Evaluation of a semiautomated App Store analysis for the identification of health apps for cardiac arrhythmias

While some applications address healthcare professionals and serve as educational or disease-management tools, others are intended for patients [1]. Targeting patients, the opportunity to remotely derive information on the patient’s heart rhythm, which enables the embedment into telemedicine treatment approaches, raised interest in the utilisation of health apps in the field of cardiac arrhythmias [9]. However, concerns about quality and safety exist and comprise loss of data privacy, poor data management, misdiagnosis by unvalidated sensors, and lack of evidence for improving medical endpoints [6]. Beyond this, due to the considerable increase in available applications in recent years, it remains difficult to find applications that meet one’s needs and fulfill quality claims [2].

Thus, methods are required to identify commercially available health apps and obtain information on purpose, target group, costs, certification as a medical product, and other quality distinctions [2]. Recently, we developed and published a semiautomated retrospective App Store analysis (SARASA) to identify and characterise health apps available on Apple’s (Cupertino, CA, USA) App Store and applied it to data from the German storefront [2, 3]. However, there have been some changes to the extent of the apps listed on the App Store’s overview pages since the inception of the methodology, necessitating an expanded approach of SARASA for obtaining a more comprehensive list of apps than would have been possible with our initial read-out methodology. Here, we present the first application of this novel approach for identifying health apps in the context of cardiac arrhythmias from Apple’s German App Store and their assignment to predefined categories.

The read-out was conducted by accessing the alphabetical listings of the applications provided by Apple on the country-specific websites of the App Store by using a script-based approach via the “iTunes Search APIs” [21]. Surprisingly, some known apps were not included in the initial read-out. Thus, we modified our method to obtain a more comprehensive dataset by separately parsing the alphabetical pages for apps starting with lower case letters and special characters, followed by an additional evaluation of the manufacturer’s store pages (stratified by device categories such as iPad or iPhone) of those manufacturers with at least one listed app in the initial read-out. For the results obtained from the manufacturers’ pages, we also had to filter out non-health apps, as these pages also listed entries for other store categories. Using the language detector of the cld2-bibliography [13], we excluded apps that did not have German or English as the main language in their description texts.

The initial read-out of the category “Medical”, obtained between November 30th (10:21:25 p.m.) and December 3rd 2022 (9:00:08 p.m.), found 28,970 apps. By parsing the manufacturer pages, we identified another 2594 apps which resulted in overall 31,564 applications in the “Medical” category at that time. Of those, for 22,713 apps (72.0%), the primary category was “Medical”, while for 8849 apps (28.0%), “Medical” was only assigned as a secondary category. By considering only those apps with German or English description texts, 22,674 apps remained for further analysis. After applying the German and English language search terms, exclusion of duplicates, as well as utilisation of the above-mentioned exclusion terms, 479 apps (2.1%) remained with a supposed context in cardiac arrhythmias. The number of hits per search term is shown in Table 1. After the manual review process, a further 44 apps were excluded since they were mismatches and unrelated to cardiac arrhythmias.

The remaining 435 apps were manually categorised according to the above-described scheme (Table 2). In all, 354 apps (81.4%) were found to deal with education, decision support, or disease management, and 114 apps (26.2%) (additionally) provided the opportunity to derive information on the heart rhythm (Fig. 1). Most of the apps were identified to be intended for healthcare professionals (243 apps; 55.9%; Fig. 1). Students and patients were addressed in 76 (17.5%) and 69 (15.9%) apps, as indicated by the description texts, respectively. In 137 apps (31.5%), the target population was not specified in the description texts. We found 108 apps (24.8%) providing a telemedicine treatment approach. Most of the description texts did not reveal any information on medical product status (364 apps; 83.7%). Based on the information provided in the description texts, we found 36 apps (8.3%), that stated to have medical device status, and 35 apps (8.0%) explicitly denying this status (Fig. 2). The median length of the description texts was 1410 characters, including spaces and characters related to formatting (min.: 65; max.: 4021). Of the 435 apps related to cardiac arrhythmias, 311 (71.5%) were free of charge. The median cost for the other apps was 5.99 € (Min.: 1.19; Max.: 1199.99).

A total of 143 apps (32.9%) had customer ratings. These had a median count of five ratings (Min.: 1 rating; Max.: 89,069 ratings). The median rating score of apps with at least one rating was 4.43 (out of 5 achievable). The top-rated health apps for the specific categories are shown in Table 3. Of note, those apps having at least one evaluation were firstly sorted to the absolute rating counts, and only the apps of the upper quartile or upper median (latter in case of < 10 apps related to the upper quartile) were considered for the ranking. Our methods and findings are summarised in a graphical illustration in Fig. 3.

The technical advances of the past decade led to tremendous growth in the mobile applications market. In the past, there were efforts to gather the full extent of apps provided for the common mobile platforms and develop tools for further characterisation concerning particular features [2, 4, 11, 19]. However, those efforts were often limited to incomplete read-outs and imprecise classifications [2, 4, 11, 19]. Via the novel, expanded approach for our semiautomated method SARASA, we were able to identify overall 31,564 apps in the “Medical” category in Apple’s German App Store. We established an automated identification of apps related to cardiac arrhythmias based on search terms and found 479 apps. Of those, only 9.2% were found not to fit the purpose, necessitating manual exclusion. This indicated a good accuracy of the search algorithm and a proper selection of the search terms. As a result, a surprisingly high number of 435 apps remained in the specific context of cardiac arrhythmias.

Whether healthcare professionals or patients, users commonly rely on the information from the description texts when choosing a suitable app. However, we found that 31.5% of the description texts did not even specify the target population. This may result in misuse of these applications on the one hand or in users refraining from downloading an app on the other hand.

Besides, many of the applications identified aimed to be instantly integrated into patient care processes, thus meeting the definition of a medical product. For the European Union, such applications have to provide evidence that the requirements of the Regulation (EU) 2017/745 of the European Parliament have been fulfilled [14]. After completing the conformity assessment, the manufacturers can attach the CE certificate to their applications [14]. In the United States, applications are approved as medical products by the U. S. Food and Drug Administration (FDA) [8]. The designation, whether a health app has been approved as a medical product or not, is highly relevant for the users and should be disclosed on the download platforms. However, according to other publications, the proportion of apps providing information on the medical product status in the description text was negligible [3]. In our analysis, only in 16.3% of the description texts did the manufacturers provide information about whether the app had medical product status (or not).

Albeit some may consider the CE certificate and FDA-clearance as quality features for medical products, these labels are rather indicators of compliance with conformity requirements that allow for market participation of medical products. They are not to be seen as quality assessments [3]. There have been some efforts to further evaluate health apps regarding quality issues. Previously, we and others worked out quality criteria for software in the context of health apps [1, 5, 15, 16]. Nevertheless, the manufacturers of medical apps are still not asked to follow a standardised quality assessment. In our opinion, an EU-wide standard is required to allow for objective and reliable quality evaluation of medical apps [1]. Due to the lack of such evaluations, users often rely on the average customer ratings to choose an application that fits their purposes. This must be seen in a critical light, but ratings are one of the few information points that can be used in coming to a decision. To provide some clinical implications, we identified the top-rated apps for the most common purposes in the context of cardiac arrhythmias (Table 3).

Nowadays, the technical advances in the field of ECG and PPG allow for remote monitoring of a patient’s heart rhythm by using smartphones or smartwatches alone or in combination with coupled sensors [9, 17]. Such applications may be helpful in diagnosing rhythm disorders in symptomatic subjects, for screening, or to follow-up patients after receiving antiarrhythmic therapy [10, 17]. Health apps for diagnosing cardiac arrhythmias are increasingly accepted [12]. In a recently conducted survey, physicians predominantly saw the advantages of using wearable rhythm devices in daily practice [12]. Although the cardiological societies have published clinical advisories for health apps in the context of cardiac arrhythmias, they avoid explicitly recommending certain manufacturers and products [10, 17]. Due to the high number of 114 apps identified in our analysis to (additionally) provide the opportunity to derive information on the heart rhythm, it is challenging for physicians to maintain an overview over which applications are approved as medical products and fulfill the required quality criteria for the particular purpose.

Our study has some imitations. Firstly, the read-out was limited to Apple’s App Store, and thus, applications only offered on other platforms are missing. Secondly, even with our expanded read-out methodology, some apps were known to be available in the store but were still missing in our acquired dataset (e.g., Fibricheck, Qompium Inc. Hasselt, Belgium). This may be due to several factors. On the one hand, Apple seems to include only apps conforming to specific (unknown) criteria on the store overview pages, probably related to an app’s performance on the store. Similarly, even via the manufacturers’ store pages (stratified by device category), there may be apps for the “Medical” category that we were unable to find. These manufacturer pages only list up to 100 apps per device category, even for manufacturers with considerably more apps in the initial read-out. Besides, there may be some manufacturers with no apps at all in the initial read-out, and we may thus have missed apps for those manufacturers as well. The high market dynamics and the associated fluctuation of apps on offer during the read-out process may also be aggravating factors. Due to the long duration for the complete read-out (aside from network speeds, also attributable to limitations in the number of requests allowed per minute by Apple’s servers), there were a few apps specified in the original lists (obtained from the overview pages), but that were missing for the metadata read-out later on. For our read-out, this was true for two apps. Thirdly, some health apps in the context of cardiac arrhythmias may have been missed due to not matching any of the keywords chosen to identify eligible apps.