The term transgender and gender diverse (TGD) refers to individuals whose gender identity does not align with their sex assigned at birth and includes individuals who identify as transgender, gender nonbinary, gender diverse, or gender-fluid [1]. The term transgender can be viewed in contrast to the term cisgender, which is used to describe those whose gender identity aligns with their sex assigned at birth.

The social determinants of health and structural stigma create health barriers for the TGD population. For example, 29% of the respondents of the 2015 US Transgender Survey lived in poverty compared with 12% of the US population [2]. In addition, the unemployment rate of respondents was 15% compared with 5% in the general US population [2]. As a result, the TGD community experiences many barriers to achieving health and well-being. Relative to cisgender individuals, transgender individuals experience disparities and inequities in all aspects of health (ie, mental, physical, emotional, and social), including poor overall health status, access to quality health care, and mental health, as well as an increased risk of substance use disorder, myocardial infarction, and sexual and reproductive health concerns [3-7]. Transgender individuals commonly experience trauma, abuse, and violence throughout their life span, leading to chronic stress [8,9]. Persons who are TGD experience many social determinants linked to poor health, including a lack of stable income and quality housing [10-12]. Stigma [13,14], ignorance [15], and discrimination contribute to poor health care access and poor care quality. Lack of social support, ranging from social exclusion and marginalization to poor social and family relationships, including rejection and family violence, contributes to stress and mental illness, including suicide [16]. Persons who are TGD are often poorly treated in the health care system and have difficulty finding TGD-competent and knowledgeable providers [15]. As a result, persons who are TGD may avoid routine care and are less likely to receive preventive services [17], less adherent to life-saving therapy [18-20], and more likely to experience denial of services from primary care to end-of-life care [21,22] and to be uninsured or have public insurance than persons who are cisgender [23]. In 2015, the American College of Physicians advocated for the creation of policies that would advance health equity among the lesbian, gay, bisexual, transgender, queer or questioning (LGBTQ) community—including the need for ongoing research on best practices for equitable health care [24]. Complicating the need for TGD-competent care is the fact that persons who are TGD experience unique needs related to gender-affirming health care as well as general health care and that their needs evolve across their life span, from youth and adolescence through midlife and late life [13,25-27].

The existing literature shows that persons who are TGD and their caregivers most often seek information to (1) explore gender identity and coming out, (2) fill health and medical knowledge gaps, (3) seek support networks, (4) find TGD-competent providers, (5) find legal advice, and (6) find advocacy/political advice [28,29]. Legal issues important to health and well-being include protections, such as health insurance discrimination, and policies and procedures for changing gender information on legal forms of identification such as a driver’s license [30]. Persons who are TGD also seek information to build skills for communication with health care providers [25,31] and information regarding strategies and counseling services to build resilience, improve body image, and contend with other stressors [32]. In addition, the TGD community needs specific medical and health information regarding gender-affirming care, including surgery, mental health support, and hormone treatments and their impact [28,29,33-37].

Barriers to finding relevant health information include (1) a general lack of TGD health information [35], (2) not knowing the terms to use when searching [35], (3) often finding hateful content and misinformation [34], and (4) identifying credible and reliable sources of health and medical information. The TGD community relies heavily on the internet for all types of information [29]. Having access to credible sources of health information could serve to balance the increasing number of articles or posts that contain misinformation or are outdated [38,39]. Evans et al [28] highlighted the need for credible and trustworthy web-based content. Relevant TGD health information should be easy to access and broadly meet the diverse needs of the TGD community [34].

A 2012 review identified and categorized several websites created for the TGD community [40]. Within the health domain, HIV was the focus of most of the websites (n=17), followed by gender-affirming surgery (n=8), mental health (n=2), primary care (n=1), and sexual health (n=1). Over the last decade, with support from the National Library of Medicine, public and health sciences libraries have focused on cataloging information resources for the TGD community, including establishing transgender resource library collections and reference services [25,26,29,41,42]. The Lesbian, Gay, Bisexual, and Transgender Health Resources Guide of the University of Colorado Strauss Health Sciences Library [43] is one such collection.

Owing to the phenomenon of mobile apps for dating and social networking, their use among the LGBTQ community has been studied extensively to assess users’ risk of HIV and sexually transmitted infections [44-47]. A recent study by Akinola et al [36] assessed the barriers and facilitators for Black transgender women to the use of mobile app technology for HIV self-testing and remote research participation. Reported facilitators included being more engaged and having increased self-agency, whereas barriers included inconsistent access to the internet and smartphones. Radix et al [48] concluded that the use of health ITs (HITs) provides opportunities to improve the quality of care for TGD individuals. Not only can HIT solutions be designed to offer education and support addressing the social determinants of health, but the community also favors these solutions [48].

Our objective was to use participatory design methods to design a health information resource to support persons who are TGD in finding and using credible health information prioritized according to their needs. We selected a delivery method known to combat the disparities of the digital divide and known to be used by the LGBTQ community already—mobile smartphones [44-47,49,50].

This section describes the design and development of our transgender health information resource (TGHIR [tigger]) [51,52]. The TGHIR platform consists of three main components (Figure 1): (1) curated health and wellness information content (henceforth, TGHIR resources), such as websites, documents, videos, and other consumer health apps; (2) a back-end system, including a database to store user data and TGHIR resources, a search engine, and a communication platform that allows users to send messages to the development team (eg, comments/feedback, suggestions of new content, and reports of offensive or inaccurate content); and (3) a front-end mobile app (henceforth, TGHIR app) that allows users to create accounts, search for information, and access/view the curated TGHIR resources. We did not create any of the TGHIR resources and, instead, incorporated links to freely available resources that can be accessed via the internet. The TGHIR resources were used as input to populate the database and as seeds to create the search engine.

We based the design and development of the TGHIR app on a conceptual framework (Figure 2) that recognized the importance of participatory design, which is a form of user-centered design that focuses on designing with end users and not merely for end users [53]. Participatory design prioritizes users and allows for their direct participation in the design process through decision-making, going beyond the role of users as consultants. Direct involvement in the entire design process often leads to greater satisfaction with both the process and the outcome [51,52]. Participatory design also represents a key strategy for designing for dissemination, sustainability, and equity, attending to potential factors that may influence widespread adoption and equitable access to the TGHIR app [54].

We applied the basic stages of participatory design by Spinuzzi [55]—stage 1: initial exploration of work; stage 2: discovery processes; and stage 3: prototyping—and implemented a participatory design process described by Schnall et al [56]. We applied the 4-phase approach by Schnall et al [56] to the design and evaluation of the TGHIR app (Figure 3). In this paper, we sequentially summarize the methods and results by phase as the results from subsequent phases inform later phases.

The project was approved as exempt human participant research by the Colorado Multiple Institutional Review Board (protocol 19-1562).

Phase 1 was dedicated to establishing a relationship between the CAB and the research team, eliciting CAB member insights into the proposed methods, and identifying TGD-related health information needs and sources.

In phase 4, test users installed and used the released TGHIR mobile app on their smartphones for a period of 4 weeks and participated in 3 evaluations. First, we conducted cognitive walk-through interviews within 2 days of the TGHIR app being installed on the test users’ phones to understand the performance of all the features. Test users were asked to try the same 10 features/tasks that the experts had evaluated (Table 2). To assess feature usability, test users were asked to respond to items 3 and 8 on the SUS [85]—I thought the feature was easy to use and I found the feature very cumbersome/awkward to use, respectively [86]—for each of the 10 features via a REDCap survey. The SUS is unidimensional and only measures 1 construct, that is, perceived usability. It has been shown that collecting responses to items 3 and 8 is 96% accurate in assessing system usability while also decreasing participant burden [86].

Test users completed the user version of the Mobile App Rating Scale (uMARS) via REDCap at 2 and 4 weeks after app installation (T1 and T2, respectively). The uMARS is a widely used scale for evaluating the quality of mobile health (mHealth) apps [87]. The uMARS is a modified version of the Mobile App Rating Scale (MARS) that is designed to be used by end users of a mobile app without training or expertise in mHealth technology or in the related health field. The uMARS contains 16 items assessing 4 dimensions of objective quality—engagement, functionality, esthetics, and information—and includes 4 items for the dimension of subjective quality. We also used 3 optional items for the dimension of perceived impact. All uMARS items are assessed on a 5-point scale (1=inadequate, 2=poor, 3=acceptable, 4=good, and 5=excellent) [87]. The MARS and uMARS have been shown to have excellent internal consistency, with high individual Cronbach α values for all subscales, and are valid, reliable, and accurate in measuring health app quality [87,88]. The MARS scores have also been shown to correlate with mHealth app revenue, monthly active users, and user downloads [89].

Through the process of participatory design [55], we successfully cocreated a TGHIR delivered as a mobile app for and with people who are TGD to provide access to credible health information resources. The participatory design process was guided by our conceptual framework, the multiphase methodology by Spinuzzi [55], and principles of community engagement. A partnership with the CAB and insights from research participants confirmed the TGD community’s need for accessible and credible health information. It also illustrated their frustrations and barriers to obtaining credible health information. Barriers to obtaining credible information include a lack of knowledge by clinicians; the lack of information on the internet and the inability to find and identify it even when it does exist; and, finally, the lack of empirical evidence for many of their questions. For the TGHIR app to be of value, it had to achieve 2 goals. First, the app should direct people to credible, useful information, and second, the app should be easy to use and satisfying to the user. After 4 weeks of use, users gave the TGHIR app an overall objective quality rating of 4.25, a subjective quality rating of 3.75, and a perceived impact rating of 4.45 out of 5.00 on the uMARS scale, which equates to a good-quality mobile app. Most importantly, the information category had the highest uMARS score of 4.75 out of 5.00. The uMARS scores indicate our success with both goals: a usable app and valuable information. The overall quality rating improved during the 2 weeks of use, suggesting that users gained familiarity with the app features. Mobile apps that have overall uMARS ratings of good quality are more likely to be adopted by the intended users than poor-quality apps [87,89]. Our 4-week engagement score was 3.98, and although this did increase from week 2 to week 4, it is not clear how this score would have changed over a longer time period. A review of publicly available asthma apps found that app quality varied and that engagement was often the lowest-scoring dimension [90]. Our TGHIR app scores were better than the asthma app scores that had average objective and subjective quality scores of 3.17 (range 1.54-4.55) and 2.65 (range 1.00-4.50), respectively. Our scorers were also the end users themselves, which may be better correlated with end-user use.

The intensity of engagement is interesting to consider for mHealth apps that seek to change user behavior, such as apps to help patients manage chronic diseases, where long-term and frequent engagement may be critical to improve outcomes [91,92]. Long-term engagement with diabetes management apps has been limited [93], and experts have suggested that virtual coaching [94] along with wellness and chronic disease app use may enhance engagement and outcomes. A recent paper exploring the value of mHealth apps for patients and how they can be incorporated into traditional medical care delivery first classified apps into four categories: (1) aiding diagnosis or decision-making, (2) improving outcomes through better disease management, (3) stand-alone digital therapeutic devices, and (4) primarily delivering education. [95] The TGHIR app falls best into category 4, as it was designed to provide information to the TGD community, and in that sense, it may assist with decision-making (ie, Should I go on testosterone or not?) and condition self-management but only by providing information and improving one’s knowledge. Information is essential for education, but we did not create specific educational materials, although some resources available through our app may have been developed with educational intent, nor was the TGHIR app a diagnostic or decision support app for a specific concern, such as whether a skin lesion is cancerous or to provide triage advice.

A recent randomized controlled trial of an mHealth app designed to deliver information to educate patients regarding knee pain showed that it increased a patient’s disease-related knowledge [96]. Others have also suggested that informational/educational apps may lead to improved processes and outcomes for diseases and conditions such as heart failure, inflammatory bowel disease, recent tonsillectomy, and various cancers [97-99]. Others have proposed that informational/educational apps may lead to improved patient-provider communications and improved decision-making, which is something that we would like to assess in future research.

The TGHIR app benefited from a research team that included clinicians who were experts in evidence-based transgender health care and a medical librarian who could search, identify, and deem credible the information resources made available via the app. Manually maintaining credible health information is a resource-intensive endeavor. Automated methods to find information on both peer-reviewed medical manuscripts and non–peer-reviewed materials would be valuable, but this also needs to be paired by credibility assessments and methods to tag content so that the TGHIR content can be searched for and found by users. Community crowdsourcing to create useful and credible health information resources has been found to have more reliability when professionals or experts are present in the process of content creation. Many concerns exist regarding the expertise (ie, no medical or health education expertise) and intent of the authors (ie, want to sell a product) [100,101]. Whether community users can judge credibility is also debated. There is research suggesting that people often report judging source credibility, but observational studies suggest otherwise [102-104]. Owing to concerns expressed with finding credible information and concerns about author intentions, we did create a contact us feature allowing users to recommend additional resources and removal of content deemed noncredible or offensive. We built a dedicated PubMed or MEDLINE search for use with the MedlinePlus application programming interface [105] and a search feature for ClinicalTrials.gov, but the prebuilt searches were not incorporated into the final TGHIR app because of time constraints. A link to the “Trust It or Trash It” resource [68] was also not incorporated into the prototype app. Both are planned for future versions pending additional funding.

There is a clear need and value proposition for the TGHIR app. The information provided by the app is not intended to replace competent clinical TGD health care and is intended to assist laypersons so they have the information needed to be engaged and informed participants in their health and well-being.

Some participatory methodological insights were generated during the project and may be helpful to other researchers. Our perception was that the TGD participants were grateful for the opportunity to take part and be involved in something that was being designed for them and with them. Several participants went beyond our requested involvement to offer advice on graphic design and other areas in which they had expertise.

Limitations included a focus on users based in Colorado and challenges in recruiting and, thus, co-designing with racially and ethnically diverse individuals. Our medical librarian did identify health information resources in Spanish, and our back-end health information resource database is designed to store the language of the resource, but we had limited resources for providing the TGHIR app interface in languages other than English and providing access to non–English-language resources. Despite striving to recruit demographically diverse co-design and end-user testing participants, we had difficulty recruiting TGD people of color and from rural settings. Although 34% (11/32) of our participants indicated that they were Black or African American, Asian, Native American, and of multiple races, we were not able to recruit any Black or African American test users. We can only hypothesize why this was difficult, including reasons such as Colorado’s demographics and the multitude of political issues necessitating this community to be active advocates at the time of this project. We expect that the TGHIR app would be useful to people of color and from rural settings regarding the user interface and functions, but it remains a question as to whether the information would have been rated as highly by these underrepresented groups. Some of the information resources were local TGD resources, which were more likely to be in the Denver metropolitan area and, therefore, might be deemed less valuable to those residing outside the metropolitan area. It is also true that rural areas are less likely to have the same extent of LGBTQ+ resources as urban areas, which is not a fault of the TGHIR app. The age distribution of our participants, 18 to ≥50 years, is a strength of this study.

Using methods of participatory design with the TGD community and in partnership with a CAB, we were able to co-design and develop a health information resource delivered via a mobile app for persons who are TGD and their care partners. Users felt that this app would be beneficial to them and that it provided needed information. A health information app is only as good as the information it makes accessible, and ongoing updating and maintenance of information resources in any information app is a challenge. Next steps include work to automate/semiautomate methods to identify relevant and credible information and testing for clinical effectiveness, including outcomes such as more engaged and useful interactions with health care providers and being better informed of the options available and their risks and benefits to support informed decision-making.