search for




 

Systematic Search and Qualitative Evaluation of Dietary Supplement Mobile Applications: Using the Mobile Application Rating Scale (MARS)
Korean J Clin Pharm 2023;33(1):51-61
Published online March 31, 2023
© 2023 Korean College of Clinical Pharmacy.

Hyeon Ji Lee1,2, Si Hyun Seong3, Hyunjin Chung3, Yun Jeong Lee1,3*, and Jae-Hyun Kim1,4*

1Institute for Digital Life Convergence, Dankook University, Cheonan 31116, Republic of Korea
2National Cancer Control Institute, National Cancer Center, Goyang 10408, Republic of Korea
3Department of Pharmacy, College of Pharmacy, Dankook University, Cheonan 31116, Republic of Korea
4Department of Health Administration, College of Health Science, Dankook University, Cheonan 31116, Republic of Korea
Correspondence to: Yun Jeong Lee, PharmD, PhD, College of Pharmacy, Dankook University, 119, Dandae-ro, Dongnam-gu, Cheonan-si, Chungcheongnam-do, 31116, Republic of Korea
Tel.: +82-41-550-1445; Fax: +82-41-559-7899; E-mail: yunlee@dankook.ac.kr

Jae-Hyun Kim, PhD, Department of Health Administration & Institute for Digital Life Convergence, Dankook University, 119, Dandae-ro, Dongnam-gu, Cheonan-si, Chungcheongnam-do, 31116, Republic of Korea
Tel.: +82-41-550-1472; Fax: +82-41-559-4800; Email: jaehyun@dankook.ac.kr
Received December 17, 2022; Revised January 16, 2023; Accepted January 30, 2023.
This is an Open Access journal distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Background: Mobile applications (apps) on dietary supplements can increase consumers’ access to information. However, it can lead to indiscriminate use of dietary supplements. This study aims to systematically review dietary supplement apps released in English and Korean and evaluate the quality of those apps. Methods: Through the app stores, apps on dietary supplements were systemically searched and examined. Two independent evaluators evaluated the apps and presented a mean score using the Mobile App Rating Scale (MARS). The correlation between MARS scores, user and evaluator ratings, and the number of secondary features of the apps were analyzed. Results: Of the 2,772 dietary supplement apps identified, 17 apps were included according to the selection criteria. The mean MARS score was 3.28 (standard deviation: 0.29) out of 5. Apps had higher scores in aesthetics and functionality dimensions, while engagement and information dimensions had lower scores. There was a positive correlation between the number of app downloads and information among MARS dimensions. The subjective evaluation also correlated with the information dimension. There was a positive correlation between the secondary features of the apps and MARS total score as well as the engagement dimension. Conclusion: The dietary supplement apps need to be managed at a higher level of quality to provide safe and reliable information to consumers. Especially, quality on information and engagement dimensions can be improved. Involvement of healthcare professionals in the app development, management with adequate referencing of information, and use of secondary features for enhanced user engagement can be helpful.
Keywords : App review, dietary supplement, MARS, mHealth, mobile applications
Body

As interest in well-being increases and health care in daily life becomes important to lead a healthy life, dietary supplements market is also increasing globally.1) Since the emergence of coronavirus disease (COVID-19), increased awareness of health and the spread of self-medication trends are expected to accelerate the growth of the dietary supplements market and further expand it.2-3) The size of the dietary supplements market in Republic of Korea has also continued to grow at an annual mean growth rate of 12% since 2016.4)

The definition, scope, and regulations on dietary supplements vary by country. Dietary supplements are defined by the US Food and Drug Administration as products taken by mouth that contain dietary ingredient, which are vitamins, minerals, amino acids, and herbs or botanicals, as well as other substances that can be used to supplement the diet.5) They are also referred to as “health functional foods”,6) “food supplements”,7) “natural health products”,8) “food with health claims”,9) among many others. While dietary supplements can help supplement the diet, obtain vital substances the body needs to function, or help reduce the risk of disease, it should be cautioned that they are not intended, and therefore should not be marketed as to treat, diagnose, prevent or cure diseases.10)

While dietary supplements are perceived to be safe by consumers as they are “natural” or “herbal”, functional ingredients in dietary supplements that provide physiological or pharmacological effects can result in adverse effects or drug-drug interactions, and this risk can be further enhanced in special populations, such as children, pregnant women, patients with impaired organ function or immune function.11-15) Although dietary supplements are available for purchase under the counseling of pharmacists at local pharmacies, as dietary supplements are readily available for purchase in stores or the internet without much restrictions, in most cases, consumers obtain information from various sources and make selections according to their perceived needs.

Along with the expansion of the dietary supplements market, the mobile health (mHealth) market is also growing at a significant rate.16) mHealth refers to devices and behaviors for health management using mobile devices such as smartphones. In particular, in an era when facing each other has become difficult due to COVID-19, health management through mobile applications (apps) is receiving more attention and is being utilized more frequently among patients and the general public.17,18) mHealth apps can be broadly classified into two categories: (a) medical apps, which are used for purposes of prevention, diagnosis, or treatment of diseases, and (b) nonmedical apps, which are those relevant to lifestyle, fitness, and well-being.19) mHealth apps for dietary supplements can be a convenient way to obtain information on dietary supplements’ health claims, compare ingredients of various products, compare reviews, obtain personalized recommendations, as well as to manage medications and intake of dietary supplements. Such apps on dietary supplements would mostly fall into the category of nonmedical apps.

However, the quality of the mHealth apps is not guaranteed. In the app market, there are few restrictions on the app development, and as the development process is not complicated and the development cost is relatively low, the market entry barrier is rather low.20) While several countries have developed regulations for high-risk medical apps, most of the nonmedical apps are not subject to such requirements.19) Therefore, it is currently up to the individual app users to appraise and properly identify apps that provide reliable and high-quality information and function. Since apps for dietary supplements can be directly related to the health and well-being of the app users, the need to ensure their quality and accuracy of the information and functions provided is even greater. In the meanwhile, a considerable number of apps on dietary supplements have been developed and are readily available to consumers without formal evaluation or peer review.

Therefore, this study aimed to systematically review dietary supplement apps released in English and Korean on the Google Play Store and the Apple App Store to examine major contents and conduct a quality evaluation of the currently available apps on dietary supplements using the Mobile App Rating Scale (MARS). Furthermore, we also aimed to evaluate the factors related to the quality of the apps and to explore the directions for enhancement of dietary supplement apps.

Methods

Systematic Search Strategy

This study was conducted by referring to previous studies that designed a systematic review of the mHealth application program by applying the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) tool for systematic review and meta-analysis to the app database.21,22) Mobile phone apps on dietary supplements were systematically searched on Google Play Store and Apple App Store. Google Play Store search was conducted on a computer after all Google accounts have been logged off to eliminate the possibility of the results being affected by the account. Apple App Store search was conducted on iPhones. The following search terms were used in English and in Korean: “dietary supplement”, “health functional food”, “nutritional supplement”, “vitamin”, “natural medicine”, “natural products”, “herb”. The search was conducted between July 19, 2021 and August 05, 2021 by two researchers.

App Selection

The inclusion and exclusion criteria of the apps were defined a priori and are summarized in Table 1. In brief, apps that were searchable under the aforementioned search terms, those available in English or Korean language, and those with last updated date within 2 years of the search were included. On the other hand, apps aimed only at dietary supplement product sales, not relevant to dietary supplements, that do not provide information on dietary supplements (i.e. information on herbal plants only), with information only for non-human (i.e. animal) use, developed only for medication administration reminder function, intended only for healthcare professional use, or requiring subscriptions outside of the app for use were some of the main exclusion criteria.

App selection criteria

Inclusion criteria
1. App retrieved on Google Play Store or iPhone App Store by the following search terms: dietary supplement, health functional food,
nutritional supplement, vitamin, natural medicine, natural products, herb
2. App available in English or Korean language
3. App that is last updated (or launched) after July 2019 (within 2 years of the search date)
Exclusion criteria
1. App aimed only at product sales (i.e. App that includes ~ Store, ~ Shop, ~ Shopping, ~ Mall, ~ Market, etc in the app title)
2. App that is not relevant to dietary supplements, does not provide information on dietary supplements (i.e. information on herbal plants only),
or only provides information on non-human use (i.e. animals)
3. App that simply serves as medication administration reminders
4. App that does not run or is not downloadable after three separate attempts
5. App that includes dietary supplement information at a minimum and/or information other than dietary supplement is mainly provided
6. App available in languages other than English or Korean
7. App intended only for healthcare professionals and not for consumers or the general public
8. App with no update record or that is not updated (or launched) within 2 years
9. App that requires subscriptions outside the app to use the content
10. App with less than 100 or unknown installation counts
11. App with no user star ratings


After the full names of the apps have been collected through the systematic search using Excel 2016, duplicate apps were removed. Then, two researchers independently screened through all the apps and excluded apps that did not meet the inclusion and exclusion criteria based on the title and description of the app on Google Play Store or Apple App Store. Any apps that were not searchable through the App Stores were also excluded. Any disagreements on the inclusion of the apps after the initial selection process were evaluated and determined by a third researcher. Apps that were included after the initial selection process were installed on either Samsung Galaxy Note 10 for Android-based apps or iPhone X and iPhone SE2 for iOS-based apps. Two independent researchers evaluated the installed app for the inclusion and exclusion criteria. If agreement was not reached between the two researchers, a third researcher independently evaluated the installed app and three researchers reached a consensus through discussion. Through the discussion, additional exclusion criteria were later added, which were to exclude apps with less than 100 or unknown installation counts, and those with no user ratings (exclusion criteria 10 and 11 on Table 1), as such apps may be in the early development stages and may not have been adequately used or tested by users. Among the 2,772 searched apps, 17 apps were included in the study.

Data Collection and Evaluation

Two researchers independently participated in the data collection process. Basic information on the app were collected, which included the app version, last updated date, number of downloads, user rating, number of raters, and price of the app. Additional information on the app were also collected, including the target group of the app, affiliations, and existence of advertisements. The secondary features provided by the dietary supplement apps were examined in the following seven features: customized dietary supplement recommendations, login function, share function, existence of app community within the app, pill taking reminder function, recording of current medications, and product comparison service. Apps were also assessed for referencing of the information provided and involvement of healthcare professional(s) in the development of the app.

App Quality Assessment Using MARS

The included apps were evaluated for quality through MARS.20) MARS is a multidimensional, expert-based mHealth app quality rating scale which has been utilized in various mHealth fields.22-27) MARS evaluates mHealth apps based on 17 items in four objective dimensions; engagement (5 items), functionality (4 items), aesthetics (3 items), and information (7 items). The rating scale utilizes a 5-point Likert scale from 1 (inadequate) to 5 (excellent), and N/A is considered valid in specified items that are considered not applicable. In addition, there is a subjective quality assessment section where the evaluators subjectively evaluate the app on 4 items (app recommendation intention, expected frequency of app use, intention to pay for the app, and overall star rating of the app). Two independent evaluators majoring in pharmacy evaluated the contents of the final selection of the apps according to the evaluation tool from August 27, 2021 to September 2, 2021. Prior to the assessment, three researchers completed training on the use of MARS28) and conducted pilot exercises on three apps to ensure the evaluation results were consistent between the raters. Through the pilot exercises, it was determined that a more objective standard was necessary for MARS items #15 (quality of information) and #16 (quantity of information), which were regarding the information provided on dietary supplements. For consistency, the researchers selected five common dietary supplements and evaluated the quality and quantity of information provided on these supplements: vitamin D (also referred to as cholecalciferol or ergocalciferol), milk thistle (also referred to as silymarin), coenzyme Q10 (also referred to as ubidecarenone), echinacea, and ginkgo (biloba or leaf). The quality and quantity of information was determined in comparison to information provided by the Korean Ministry of Food and Drug Safety29) and/or Korean Integrated Medical Information System (KIMS),30) a Korean drug search engine. Two researchers independently participated in the MARS rating, and a third researcher was involved in cases of uncertainty, which was resolved through discussion among the researchers. Mean MARS score between the two researchers were used for final analysis.

Statistical Analysis

After calculating the mean value for each MARS item, the overall app quality score was evaluated by calculating the mean value for each of the four objective dimensions, and the overall subjective app quality was evaluated by calculating the mean value of the items in the subjective dimension, as recommended by the developers.23) In addition, correlation coefficients were calculated to examine the correlation between the MARS score and user star rating, number of raters, number of downloads, evaluators’ subjective evaluation, and number of secondary features. A p-value less than 0.05 was considered statistically significant. The Cronbach’s alpha value of this study was 0.90. All analyses were performed using the SAS version 9.4 (SAS 9.4, Cary, NC. SAS Institute Inc.).

Results

Results of App selection

A total of 2,772 apps were identified through search terms in the app database, which included 2,372 apps on the Google Play Store and 400 apps on the Apple App Store. 690 duplicated apps and 94 apps that were not retrievable on app stores were excluded. A total of 1,988 apps were screened through the inclusion/exclusion criteria, and 1,947 apps were initially excluded. Forty-one apps were downloaded for further evaluation of the inclusion/exclusion criteria, and 24 apps were additionally excluded. Finally, 17 apps were included in the study (Fig. 1).

Fig. 1. Flow chart of the application selection process.

Characteristics of included apps

The characteristics of the apps included in the analysis are as follows. Of the 17 apps, 12 apps were compatible with both Android and iOS, whereas five apps supported Android only. All the apps clearly identified the developer and were updated at least twice. The mean user rating of all the included apps were 4.09 (standard deviation (SD): 0.56) out of 5. According to the star ratings of app users, ‘Medicinal Herbs & Supplements’ had the lowest rating of 3.2, and ‘Personalized Nutrition (Pulmuone Health & Living)’ had the highest rating with 4.9 points (Table 2).

Characteristics of included apps

App number and name Platform (Android/ iOS) Developer Involvement of healthcare professionals Referencing of information Tested app version Mean user star rating* Mean MARS score* Mean MARS subjective score* No. of secondary features
1 Drugs.com Medication Guide Both Drug.com RPh, PharmD, MD Always 2.12.1 4.6 3.81 4.33 6
2 Medicinal Herbs & Supplements Android Great_Apps - No reference 2.5 3.2 3.13 2.67 1
3 Vitamins, Minerals, Nutrients for immunity (Free) Android Medical Group Soft - No reference 3.6.6 4.5 3.10 3.67 1
4 Personalized Nutrition (Pulmuone Health & Living) Both Pulmuone Health & Living Nutritionist, Pharmacists Sometimes 1.3 4.9 3.12 2.50 2
5 Health Keeper Both Minwise Co., Ltd - No reference 1.1.0 3.5 2.76 2.17 2
6 Health Charging Station Android Hndsolution - Sometimes 1.0.15 3.7 2.88 2.17 5
7 Health-one-know Both UBcare Co., Ltd Prof. in Clinical Nutrition Always 1.6.2 4.2 3.26 2.83 6
8 Guntoo (Transparent Dietary Supplement) Both Econy Prof. in Food engineering, Nutritionist, Pharmacists, etc. Sometimes 3.2 3.5 3.55 3.83 5
9 Dr.milligram Both Dr.milligram MD No reference 4.5 4.1 3.20 2.83 2
10 Drug info Android Druginfo Pharmacists No reference 4.0.0 3.8 3.15 3.00 0
11 My puzzle Both NUTRI-1 Co., Ltd. MD, Pharmacists, Ph.D. in Food and nutrition Sometimes 1.0.4 4.8 3.41 3.17 4
12 My herb Both KHERB - Sometimes 1.1.4 3.5 3.30 3.17 6
13 Ayak Both Ayak Inc. Pharmacists No reference 1.0.8 4.5 3.60 3.50 4
14 Al-yak Mul-yak (Pills and Potions) Both Howtoyak Inc. Nutritionist, Pharmacists Sometimes 1.0.5 4.7 3.48 2.83 4
15 Welcoach Android Welcoach - Always 1.0.22 4.4 2.93 2.33 2
16 Pillycare Both Carewith PharmD Sometimes 1.1.3 3.3 3.76 3.83 4
17 Homespick Both InnomaxGlobal.co.Ltd - No reference 1.1.8 4.3 3.29 3.67 6
Mean (Standard Deviation) 4.09 (0.56) 3.28 (0.29) 3.09 (0.64) 3.53 (2.00)

*Scores out of 5.

MARS, Mobile App Rating Scale



Secondary features other than providing dietary supplement information were reviewed for each app. The apps had a mean of 3.53 (SD: 2.00) features with a minimum of 0 and a maximum of 6 features among the 7 features. Login function was the most commonly provided feature (14/17 apps), followed by customized recommendations (10/17 apps), share function and recording of current medications (9/17 apps each), pill taking reminder function (8/17 apps), product comparison services (6/17 apps), and the app community (4/ 17 apps) (Appendix Table 1).

A total of 10 apps (58.8%) indicated involvement of healthcare professional(s) in the app development, in which pharmacists (8 apps), nutrition specialists (5 apps), and medical doctors (3 apps) participated. For provision of information on dietary supplements and its source of references, only 3 apps (17.7%) always provided references, 7 apps (41.2%) sometimes provided references, and 7 apps (41.2%) did not provide any references to the source of information.

Overall app quality evaluation using MARS

The overall MARS app quality score of the 17 apps was 3.28 (SD: 0.29) (Appendix Table 2), and the overall app subjective quality score was 3.09 (SD: 0.64) (Appendix Table 3). Among the four objective dimensions, aesthetics had the highest score with a mean of 3.75 (SD: 0.36), followed by functionality with a mean of 3.65 (SD: 0.39), information with a mean of 3.14 (SD: 0.53), and engagement had the lowest score with mean of 2.56 (SD: 0.51) (Fig. 2). ‘Health Keeper’ and ‘Health Charging Station’ had the lowest mean MARS app quality score of 2.88, and ‘Drugs.com Medication Guide’ had the highest mean score of 3.81. The evaluators’ app subjective quality score was also lowest for ‘Health Keeper’ and ‘Health Charging Station’ with 2.17 points, and ‘Drugs.com Medication Guide’ was the highest with 4.33 points.

Correlations between app characteristics and quality

Variables 1 2 3 4 5 6 7 8 9 10 11 12 13
1 MARS total scores 1.0000
2 Engagement 0.6361* 1.0000
3 Functionality 0.5420 -0.1053 1.0000
4 Aesthetics 0.6106* 0.4571 0.0314 1.0000
5 Information 0.7864** 0.2063 0.5506 0.2040 1.0000
6 No. of downloads 0.4469 0.0961 0.4068 0.0261 0.5818 1.0000
7 User star rating 0.1494 0.3141 -0.1827 -0.1486 0.2616 0.2432 1.0000
8 No. of raters 0.4522 0.1351 0.4007 0.0238 0.5618 0.9970*** 0.2501 1.0000
9 Subjective MARS scores 0.8348*** 0.4671 0.5437 0.2936 0.7993** 0.5310 0.0968 0.523 1.0000
10 Recommendation intention 0.8063*** 0.5903 0.5079 0.3351 0.6151* 0.3325 0.1075 0.3417 0.9010*** 1.0000
11 Expected app use frequency 0.7395** 0.4540 0.3856 0.2781 0.7269** 0.5604 0.1489 0.5466 0.9495*** 0.8124*** 1.0000
12 Evaluators’ star rating 0.7381** 0.2198 0.6191* 0.1841 0.8438*** 0.5409 -0.0082 0.5254 0.8704*** 0.6457* 0.7467** 1.0000
13 No. of secondary features 0.4942 0.7586** -0.1264 0.2892 0.2554 0.2488 0.0563 0.2837 0.3932 0.4127 0.4578 0.1763 1.0000

Mobile App Rating Scale (MARS)

***p< .0001, **p<0.001, *p<0.01, p<0.05



Fig. 2. Mobile Application Rating Scale (MARS) scores by dimensions. The overall MARS app quality score of the 17 apps was 3.28 (Standard Deviation (SD): 0.29). Among the four objective dimensions, aesthetics had the highest score with a mean of 3.75 (SD: 0.36), followed by functionality with a mean of 3.65 (SD: 0.39), information with a mean of 3.14 (SD: 0.53), and engagement had the lowest score with mean of 2.56 (SD: 0.51).

Relationships Between App Characteristics and Quality

Relationships between the overall MARS score, four MARS dimensions, app characteristics, and subjective quality dimensions were determined using correlations (Table 3). Total MARS score was positively correlated to all four dimensions in the order of information, engagement, aesthetics, and functionality. Among the four dimensions, functionality and information dimensions were positively correlated. The number of app downloads and the number of app raters had a significant positive correlation with information dimension with a correlation coefficient of 0.582 (p=0.0143) and 0.562 (p=0.0189), respectively. The user star rating could not confirm a statistically significant correlation with the MARS dimensions. The subjective MARS score was positively correlated to the following variables with the correlation coefficient in the order of total MARS score, information dimension, functionality dimension, the number of app downloads, and the number of raters. Among the subjective quality assessment items, app recommendation intention and MARS total score and all MARS dimensions except for aesthetics had statistically significant positive correlation, and the correlation coefficient was in the order of total MARS score, information, engagement, and functionality. In addition, a subjective quality item, expected frequency of app use had a statistically significant positive correlation with the total MARS score, information dimension, the number of app downloads, and the number of raters. Furthermore, the subjective star rating also had a statistically significant positive correlation in the order of information dimension, total MARS score, functionality dimension, the number of app downloads, and the number of raters. The number of secondary features had a statistically significant positive correlation with the engagement dimension with a correlation coefficient of 0.759 (p=0.0004), followed by total MARS score with a correlation coefficient of 0.494 (p=0.0438).

Discussion

This study systematically examined all dietary supplement apps released in English and Korean to extract general and functional characteristics and to evaluate the quality of the available apps using MARS. Furthermore, this study aimed to evaluate the dimensions related to the quality of the apps and to provide directions for improvement of dietary supplement apps. The findings have several points of discussion.

First, a gap in the ratings between app users and the MARS evaluators were observed.31) Of the 17 included apps, the mean user star rating was 4.09 out of 5, while the mean MARS app quality score was 3.28 and the subjective quality score was 3.09. This result indicates lack of high quality dietary supplement apps on the market and need for user discretion when selecting potential dietary supplement apps based on user ratings. While significant relationship between the user star ratings and the MARS dimensions were not identified, it was interesting that the number of app downloads and number of app raters had significant positive correlation with the information dimension. While the user star ratings do not necessarily correlate to the quality of the apps,20) information is an important aspect for app use, and accessibility to many high quality dietary supplement apps is necessary.

Second, among the MARS dimensions, functionality and aesthetics had high scores, and the score deviations between the apps were not large. In addition, in the correlation analysis, aesthetics did not show a significant correlation with the evaluators’ subjective evaluation results. This result indicates that it requires more than a certain level of function and interface to be released as an app. It is also the result of adding evidence that functionality and aesthetics alone cannot make a high-quality app.32)

Additionally, according to the evaluation score for each MARS dimension, engagement had the lowest MARS score, and the deviation between apps was high. Furthermore, information dimension showed the highest deviation with the next lowest MARS score. These results of engagement and information can degrade the motivation and persistence of the app use, which can hinder the app use for consumers.33) Engagement score significantly increased as the number of secondary features of the app increased, suggesting that engagement can be improved through supplementing secondary features in the app. Customized recommendation of dietary supplements based on user input of one’s health status was an interesting and commonly provided secondary features of the apps identified in this study, although the algorithm and its reliability of the accuracy of recommendations have not been evaluated. Other secondary features, such as the login function, app community within the app, recording of current medications, pill taking reminder function, and product comparison services were also commonly provided, which could increase user engagement and encourage continued use of the app.

The most notable result in this study was that information dimension score was rather low. Although the main aim of the dietary supplement apps are typically to deliver information to users, low information quality was identified in MARS-based scores. Low information scores in MARS have also been identified in several previous app contents analysis studies.22,34) For dietary supplement apps, this result suggests that apps may provide insufficient or inaccurate information on dietary supplements, potentially providing unsafe information to app users. In fact, 41.2% of the apps did not provide any reference to the source of information provided within the app. According to the mean score for each item that constitutes information in the MARS dimension, the accuracy and goals of app description were above the mean, but the quality of information, amount of information, visual information, and credibility of the source of information were below the mean. In particular, credibility of information showed the lowest mean score. Even in previous studies related to dietary supplement search using the Internet,35) low reliability of information was a problem. As the same problem occurs even when moving to a mobile environment, it seems that involvement of healthcare professionals in the development or peer review process of the app would be important in providing reliable information on dietary supplements in a digital environment. Through correlation analysis in this study, information seemed to be related to the quality of the app the most. Previous studies also suggest that the overall quality of the app can be improved by increasing information quality. Therefore, providing evidence-based and up-to-date information is essential for a high-quality dietary supplement app.34)

As a result of the study, while various dietary supplement apps are readily available for consumers, the result of the apps’ quality evaluation using MARS suggested that the overall quality of the apps were not high, and user ratings do not necessarily reflect the quality of the apps. Especially, quality on engagement and information dimensions were low, which suggest potential risk in user safety and insufficient motivation of continued use of the apps for users. Therefore, it is necessary to improve the information quality of the apps to achieve the original goal of the apps and ensure user safety, and to increase engagement quality of the apps by installing secondary features to motivate continued use of the apps.

This study is a systematic review and quality evaluation of dietary supplement apps conducted for the first time to the best of our knowledge. This study also tried to minimize bias by having two independent evaluators for app evaluation. However, there are some limitations to the study. First, since we limited the apps to those developed in English or Korean, many of the foreign apps may have been excluded from this study, potentially limiting generalizability. Second, the research was conducted over a limited period of time, and the characteristics of the apps presented in this study may be limited to reflecting the apps identified during the time of the study, as the app trends and characteristics continue to change and evolve over time.

Conclusion

The dietary supplement apps need to be developed and maintained at a higher level of quality to provide safe and reliable information to consumers. Currently available apps indicated the need to improve information and engagement dimensions to ensure safe and continued use by the users. In order to achieve this, involvement of healthcare professional(s) in the development and/or management of the apps with adequate referencing of information and adding secondary features for user engagement can be helpful.

Conflicts of Interest

The authors have no conflicts of interest to declare with regards to the contents of this study.

References
  1. Karelakis C, Zevgitis P, Galanopoulos K, Mattas K. Consumer trends and attitudes to functional foods. J Int Food Agribus Mark. 2020;32(3):266-94.
    CrossRef
  2. Onchonga D. A Google Trends study on the interest in self-medication during the 2019 novel coronavirus (COVID-19) disease pandemic. Saudi Pharm J. 2020;28(7):903-4.
    Pubmed KoreaMed CrossRef
  3. Bennadi D. Self-medication: A current challenge. J Basic Clin Pharm. 2013;5(1):19-23.
    Pubmed KoreaMed CrossRef
  4. Ministry of Food and Drug Safety. Health functional food production status. Available from https://www.mfds.go.kr/eng/wpge/m_9/denofile.do. Accessed May 11, 2022.
  5. US Food and Drug Administration. FDA 101: Dietary Supplements. Available from https://www.fda.gov/consumers/consumer-updates/fda-101-dietary-supplements. Accessed May 11, 2022.
  6. Ministry of Food and Drug Safety. Health functional food code. Available from https://www.mfds.go.kr/eng/wpge/m_9/denofile.do. Accessed February 25, 2023.
  7. European Food Safety Authority. Food supplements. Available from https://www.efsa.europa.eu/en/topics/topic/food-supplements. Accessed May 11, 2022.
  8. Ministry of Health and Labour and Welfare. Food with Health Claims, Food for Special Dietary Uses, and Nutrition Labeling. Available from https://www.mhlw.go.jp/english/topics/foodsafety/fhc/index.html. Accessed May 11, 2022.
  9. US Food and Drug Administration. What you need to know about dietary supplements. Available from https://www.fda.gov/food/buy-store-serve-safe-food/what-you-need-know-about-dietary-supplements-expired. Accessed February 25, 2023.
  10. Dennehy CE, Tsourounis C, Horn AJ. Dietary supplement-related adverse events reported to the California Poison Control System. Am J Health Syst Pharm. 2005;62(14):1476-82.
    Pubmed CrossRef
  11. Ronis MJ, Pedersen KB, Watt J. Adverse effects of nutraceuticals and dietary supplements. Annu Rev Pharmacol Toxicol. 2018;58:583-601.
    Pubmed KoreaMed CrossRef
  12. Paik DJ, Lee CH. Review of cases of patient risk associated with ginseng abuse and misuse. J Ginseng Res. 2015;39(2):89-93.
    Pubmed KoreaMed CrossRef
  13. Izzo AA, Ernst E. Interactions between herbal medicines and prescribed drugs. Drugs. 2009;69(13):1777-98.
    Pubmed CrossRef
  14. Kroll DJ. ASHP statement on the use of dietary supplements. Am J Health Syst Pharm. 2004;61(16):1707-11.
    Pubmed CrossRef
  15. Onodera R, Sengoku S. Innovation process of mHealth: An overview of FDA-approved mobile medical applications. Int J Med Inform. 2018;118:65-71.
    Pubmed CrossRef
  16. Adans-Dester CP, Bamberg S, Bertacchi FP, et al. Can mHealth technology help mitigate the effects of the COVID-19 pandemic? IEEE Open J Eng Med Biol. 2020;1:243-8.
    Pubmed KoreaMed CrossRef
  17. Linz D, Pluymaekers NA, Hendriks JM. TeleCheck-AF for COVID-19: A European mHealth project to facilitate atrial fibrillation management through teleconsultation during COVID19. Eur Heart J. 2020;41(21):1954-5.
    Pubmed KoreaMed CrossRef
  18. Organisation for Economic Co-operation and Development. Mobile technology-based services for global health and wellness: opportunities and challenges. Available from https://www.oecd.org/sti/ieconomy/mobile-technology-based-services-for-global-health.htm. Accessed February 25, 2023.
  19. Stoyanov SR, Hides L, Kavanagh DJ, et al. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth and uHealth. 2015;3(1):e27.
    Pubmed KoreaMed CrossRef
  20. Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1-9.
    Pubmed KoreaMed CrossRef
  21. Kim BY, Sharafoddini A, Tran N, et al. Consumer mobile apps for potential drug-drug interaction check: systematic review and content analysis using the Mobile App Rating Scale (MARS). JMIR mHealth and uHealth. 2018;6(3):e74.
    Pubmed KoreaMed CrossRef
  22. Siddique AB, Krebs M, Alvarez S, et al. Mobile apps for the care management of chronic kidney and end-stage renal diseases: systematic search in app stores and evaluation. JMIR mHealth and uHealth. 2019;7(9):e12604.
    Pubmed KoreaMed CrossRef
  23. Shen C, Jiang B, Yang Q, et al. Mobile apps for drug-drug interaction checks in Chinese app stores: systematic review and content analysis. JMIR mHealth and uHealth. 2021;9(6):e26262.
    Pubmed KoreaMed CrossRef
  24. Stec MA, Arbour MW, Hines HF. Client‐centered mobile health care applications: using the Mobile Application Rating Scale instrument for evidence‐based evaluation. J Midwifery Womens Health. 2019;64(3):324-9.
    Pubmed CrossRef
  25. Kalhori SRN, Hemmat M, Noori T, et al. Quality evaluation of English mobile applications for gestational diabetes: app review using Mobile Application Rating Scale (MARS). Curr Diabetes Rev. 2021;17(2):161-8.
    Pubmed CrossRef
  26. Knitza J, Tascilar K, Messner E-M, et al. German mobile apps in rheumatology: review and analysis using the Mobile Application Rating Scale (MARS). JMIR mHealth and uHealth. 2019;7(8):e14991.
    Pubmed KoreaMed CrossRef
  27. Stoyanov S. MARS training video. Available from https://www.youtube.com/watch?v=25vBwJQIOcE. Accessed August 1, 2021.
  28. Ministry of Food and Drug Safety. Korea Institute of Drug Safety & Risk Management. Pharmaceutical Integrated intelligence system: Information search for medicines. Available from https://nedrug.mfds.go.kr/searchDrug. Accessed May 11, 2022.
  29. KIMS Co, 2023. Available from www.kimsonline.co.kr.. Accessed February 25.
  30. Davalbhakta S, Advani S, Kumar S, et al. A systematic review of smartphone applications available for corona virus disease 2019 (COVID19) and the assessment of their quality using the mobile application rating scale (MARS). J Med Syst. 2020;44(9):164.
    Pubmed KoreaMed CrossRef
  31. Girardello A, Michahelles F. AppAware: which mobile applications are hot? In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, Lisbon, Portugal September 7-10, 2010.
    CrossRef
  32. Kim S, Baek TH. Examining the antecedents and consequences of mobile app engagement. Telemat Inform. 2018;35(1):148-58.
    CrossRef
  33. Bardus M, van Beurden SB, Smith JR, et al. A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. Int J Behav Nutr Phys Act. 2016;13(1):13-35.
    Pubmed KoreaMed CrossRef
  34. Ohn J, Kim JH. Intake pattern and needs assessment for the development of web-contents on health functional foods according to age of adults. Korean J Community Nutr. 2012;17(1):26-37.
    CrossRef


March 2024, 34 (1)
Full Text(PDF) Free

Social Network Service
Services

Cited By Articles