The COVID@Home program was designed to provide home monitoring of hospitalized patients with COVID-19 who had an ongoing oxygen requirement but were otherwise clinically stable. The two main components of the program were (1) inpatient screening and (2) at-home management, both of which were facilitated by the web app (Figure 2). Patient screening for program eligibility, which included a home safety evaluation, used the web app and was performed by a project nurse. If eligible, the patient was enrolled and consent was requested. Hospital discharge notes were made using the web app, and the patient was discharged with oxygen supplies and a pulse oximeter, along with appropriate education and instructions. Each day a home care provider (HCP) called the patient to check in on them and collect monitoring information using the web app. Once patients met specific clinical criteria, they were discharged from the program, the oxygen supplies were picked up from their home, and a discharge note was completed using the web app. Patients were transferred to the hospital if their condition worsened.

The target patient population was hospitalized adult COVID-19 patients in Armenia. Clinically, patients were required to have stable vital signs (including blood pressure, heart rate, temperature, and breathing rate), stable glucose levels, and oxygen saturation >93% on no more than 10 L/min. Other eligibility criteria included access to a phone, a safe living environment, and access to electricity. The program was hosted in Yerevan with patients from 7 participating hospitals: Mikayelyan Institute of Surgery, Surb Grigor Lusavorich Medical Center, the National Center for Infectious Diseases, Avan Clinic of Surb Grigor Lusavorich Medical Center, Surb Astvatsamayr Medical Center, Scientific Center of Traumatology and Orthopaedy, and Erebuni Medical Center. Patients lived either in Yerevan or in close proximity to it (<40 km).

Development of the software involved 3 clinicians, 2 program administrators, 3 software developers, and 1 project manager. All team members resided in either Armenia or the United States.

This study received internal review board approval (#CHLA-22-00028) from the CHLA.

The HCD process lasted 2 months, with design meetings involving US- and Armenian-based physicians, engineers, and project managers that were hosted over Zoom. Deliverables included a complete wireframe, a spreadsheet with Armenian language translations, and a document with user acceptance testing (UAT) feedback. Deliverables were reviewed by Armenian clinicians for suitability for use in Armenia. Comments from the UAT helped improve the user interface and user experience, such as by adding clicks, improving navigation, and adding a function to validate form input. Hard-copy backups were included at the suggestion of users.

Six biweekly meetings were held with the software developers to finalize the platform. Meeting times were adjusted to better accommodate the fluctuating and busy schedules of the clinical staff. Using the initial deliverables from HCD, the software developers created a demonstration version of the app that was validated by stakeholders through trial and error. Over 200 JIRA tickets were completed during development and testing. The final version was rolled out to end users and implemented by local stakeholders with ancillary services from US colleagues.

Five customized web pages were created: a login page, a home page, and 3 screening forms (Figure 4). Four OpenMRS modules were developed for inserting comma-separated value (CSV) data from hard-copy forms, exporting form data, editing participant status, and finding records. There were more than 100 new OpenMRS concepts created to accommodate each data point in both Armenian and English. Concepts were integrated using 2 Groovy scripts: CSV to concepts and CSV to observations. Custom drug classes were also included using 2 Groovy scripts: CSV to drug classes and concept from drug classes. A total of 52% of the code was written in Java and 47% in Groovy.

The COVID@Home program started enrolling patients in April 2021 (Figure 1). During the first few months, while the web app was still being developed, data were tracked in an online form. The web app went live in August 2021, and all records prior to that point were batch imported into the web app platform. Between April 2021 and March 2022, 439 unique patients completed 5087 forms using the web app. A majority of the forms were from daily monitoring (4430/5087, 87%), followed by screening (439/5087, 9%) and discharge (218/5087, 4%) forms. A total of 14 staff members were initially trained on how to use the web app and complete the forms. The staff received informal support from the technical team through tutorials on logging in, resetting passwords, and using the web app. Paper forms were used to collect data as needed, such as when computers or support staff became unavailable. The paper forms were later entered into the web app database. The software worked as expected, and no critical bugs or issues were reported.

Our international, multidisciplinary partnership successfully developed and implemented a free, open-source web app to remotely monitor COVID-19 patients in Armenia at home by leveraging HCD and an agile software development methodology. Developing software with international colleagues required attention to logistics and strategy. Active involvement with local stakeholders in Armenia was a priority. Due to substantial differences in time zones, virtual meetings were held early in the morning in the United States (Pacific time), corresponding to evening in Armenia. Additionally, program administrators from the United States visited colleagues in Armenia to establish rapport and facilitate stakeholder buy-in for the project when travel restrictions were relaxed. We explicitly used an HCD approach to ensure that the web app was linguistically and culturally relevant and responsive to the local technical environment and workflows. An important requirement for adapting the software for the end user was the integration of language translations for all components of the web app. We contracted the software development to a local team in Armenia and communicated with local stakeholders, which allowed us to test and iterate the web app more effectively.

We searched PubMed for publications describing COVID-19 patients who were discharged home with supplemental oxygen therapy and their remote monitoring and management using mobile or web apps with the following key words and Medical Subject Headings (MeSH) terms: (COVID-19/therapy[Mesh]) AND (Telemedicine[Mesh] OR Home Care Services[Mesh] OR Telemetry[Mesh] OR Patient Discharge[Mesh] OR telemedicine OR home OR telemetry OR remote monitor* OR virtual) AND (Oxygen Inhalation Therapy[Mesh] OR oxygen). We identified a total of 5 studies, which took place in the Netherlands (n=1), United States (n=2), United Kingdom (n=1), and Egypt (n=1) [28-32]. Aside from the Egyptian study, all the studies took place in high-income countries (HICs). This imbalance reflects the technical, infrastructural, and financial health care divide between HICs and LMICs. All 5 studies had 2 common features: collection of vital signs and communication via telephone [28-32]. Progress assessment and vital signs, such as oxygen saturation, heart rate, temperature, and blood pressure, were captured by all the studies and then stored in electronic databases, not unlike our current project. Capturing health data in electronic databases helps support research, collaboration, and quality improvement [33]. Although all the studies involved a mobile app (for education, assessment, or monitoring vital signs), they all relied on communication with the clinical team via phone calls [28-32]. Our program operated similarly, which supports the utility of leveraging standard forms of communication for remote monitoring. Three of the 5 studies conducted patient satisfaction surveys, with all of them reporting 94% or greater satisfaction with the remote monitoring program and emphasizing the benefit and utility to patients [28,29,32]. With respect to the development of these programs, only 1 described the process in detail [30]. The Atrium Health Hospital at Home (AH-HaH) program was designed and implemented rapidly in the United States with the help of numerous stakeholders with backgrounds including clinical medicine, administration, research, technology, and innovation [30]. This mirrors our process and reiterates the multidisciplinary nature of developing home-based clinical programs supported by technology. However, the AH-HaH program was implemented using existing infrastructure and resources that are not often available in LMICs [30]. Evidently, our program in Armenia was developed with a limited amount of existing infrastructure and operational resources. Other remote monitoring programs have also commented on the importance of a flexible and nimble development framework due to changes in COVID-19 guidelines, an observation that reflected our experience in Armenia as well [34].

The main strength of our project was the rapid, coordinated, and collaborative approach. With logistical uncertainty, limited knowledge, and the evolution of COVID-19 and its variants, it was important to quickly and responsively develop health screening and monitoring forms with the most recent available knowledge. This required updating form questions, the criteria, and form fields as more evidence became available. In turn, the software development cycle reflected this process and required updating code and functionality simultaneously. The conflict with Azerbaijan in 2020 had further strained the Armenian health care system, so the iterative, agile methodology allowed the team to be flexible and responsive to evolving capacity. Other strengths were the reliance on scientific evidence and the involvement of clinicians in the development process. Although case studies and literature on COVID-19 were available, the novelty of the disease and evolution of new strains led to uncertainty. Knowledge of the disease within the context of Armenia was even more limited. However, frontline workers in Armenia helped design the clinical protocols and data collection forms and were informed by their experiences and the best available evidence. This project was truly a collaborative effort, involving individuals of diverse professional experiences, cultures, disciplines, and knowledge. Each feature and change was tested by stakeholders and end users, who provided guidance on adapting the system for local use, such as suggesting language changes and workflow updates. The COVID@Home program would not have been possible without a multidisciplinary and collaborative team of clinicians, software developers, researchers, and local stakeholders. Finally, COVID@Home set a precedent for active data collection, reporting, monitoring, and outcome evaluations outside of a research setting, which is rarely done in Armenia.

A significant limitation of our approach was the reliance on hard-copy forms, which affected the quality of the data. At various points during the project, computers were only intermittently available for data entry. This resulted in patient data being recorded on paper forms instead of the web app. By the time this was brought to the attention of the project team, hundreds of patient records had been created using paper forms. Dedicated computers were provided for data entry, web app access, and digitizing the paper forms, but this resulted in several records with nonstandardized or missing data. During this time, the software continued to operate as expected. A general challenge across many global health partnerships is the long-term sustainability of a project [35]. This is a critical component of global health partnerships and is built into all of our collaborations in Armenia. For this particular project, the total number of patients enrolled in the program has drastically decreased since July 2022 as COVID-19 hospitalizations have decreased, and the plan from the Ministry of Health is to sunset this program. The technical infrastructure is maintained by a local team that supports multiple grant-funded projects, and they are committed to continue operating the existing system. As this was the first time a home health-monitoring program was used by the Ministry of Health, the ministry is currently evaluating whether to expand this approach to other use cases.

A key lesson learned from this experience was the importance of clear communication and training for all users, particularly the frontline HCPs. A more thorough needs and capacity assessment likely would have detected the issue of paper-based data collection; although the project team asked if a computer was available, additional follow-up questions may have revealed that there were limitations affecting who could access the computer, how often, and how often there were interruptions to that access. Prospective utilization monitoring might also have detected that new records were not being created, prompting further investigation. As an operational initiative leveraging existing staff and resources, the project did not operate in a vacuum, and was affected by broader factors in the health system: lack of human resources, lack of planning, and lack of proper equipment. For example, in one instance, the project coordinator was reassigned, leaving frontline staff with no support for several weeks. The development of paper forms played an important and pragmatic role in the success of the program, though once staff transitioned to paper forms due to computer access issues, it was difficult to shift the workflow back to the web app. In a less chaotic environment, not having paper forms at all may help increase the speed of adoption and prevent parallel workflows. Future projects will need to better evaluate cultural, sociotechnological, and logistical barriers to technology adoption in busy health care settings.

COVID-19 has caused tremendous clinical and logistical challenges around the globe, particularly in LMICs. Our experience in Armenia demonstrates that an international collaboration can leverage an agile methodology to rapidly develop software needed to support a home monitoring program for patients with prolonged oxygen requirements secondary to COVID-19 using free, open-source software. This project can serve as an example for at-home care in resource-limited settings like Armenia and other LMICs. Successful implementation requires a thorough needs assessment, clear and open communication, and prospective utilization monitoring to identify and address barriers.