Introduction

Oral cancer, which includes cancers of the lip and oral cavity, ranks as the 16th most common malignancy worldwide1. Incidence rates are rising, particularly among young adults2. Early prevention and treatment are critical for improving patient outcomes, as advanced oral cancer is associated with significant physical disfigurement, reduced quality of life, and financial burden3. Therefore, public education about oral cancer is essential.

Social media platforms have become vital resources for patients seeking medical information, particularly among younger demographics4,5,6. YouTube is the most widely used long-form video platform globally; however, it is not accessible in China, where Bilibili, often referred to as “Chinese YouTube”, is the leading platform. Despite their importance, no studies have compared the quality of oral cancer-related information between the two platforms. Furthermore, while recognizing the positive role of social media videos in disseminating health information, several scholars have raised concerns about the prevalence of misinformation on these platforms7,8.

This study offers the first comprehensive analysis of oral cancer-related videos on two major long-video platforms, YouTube and Bilibili. We thoroughly examine several key characteristics, evaluate the content quality of these videos, and provide useful recommendations for content creators and platform administrators.

Materials and methods

Search strategy

To minimize bias, we conducted a comprehensive search on YouTube and Bilibili on August 7, 2024. The search keywords “oral cancer”, “oral carcinoma”, and “口腔癌” were utilized on YouTube, while “口腔癌” was used for Bilibili. “口腔癌” is written identically in both traditional and simplified Chinese characters, and is consistent in academic and spoken Chinese. To reduce algorithmic influence, we created a new account for each platform. The videos were reviewed in the default order as determined by the platforms’ algorithms. The protocol for the search strategy is delineated in Fig. 1.

Fig. 1
figure 1

Search strategy for videos on oral cancer.

Full size image

Quick data collection

On August 7, 2024, the following data were collected concurrently: websites and rapidly changing metrics (views, likes, coins, collections, shares, and the number of the authors’ followers). However, it should be noted that collections and shares were not available on YouTube. “Coin” is a unique engagement metric on Bilibili, often regarded as a “precious like”, as users only receive one coin daily.

Exclusion criteria

Videos were excluded if they met any of the following criteria: ① YouTube videos in languages other than English or Chinese; ② Bilibili videos in languages other than Chinese; ③ Videos that primarily consisted of advertisements and lacked useful medical information; ④ Duplicated videos9; ⑤ Content unrelated to the medical topic, such as memorializing a celebrity diagnosed with oral cancer or fundraising for treatment. After applying these exclusion criteria, the top 150 videos from each platform were selected for further analysis.

Further data collection

From August 8 to August 16, 2024, additional time-invariant data were collected. This included titles, release dates, video lengths, author IDs, certification status, author type, video topics, video styles, and quality scores.

The certification method was based on Liu’s research9. The video authors were categorized as follows: doctors, other medical professionals, hospitals/non-profit organizations, official media, companies with profit, and self-media. The video topics were classified into the following types: etiology/prevention, pathology, symptoms, examinations/diagnosis, and treatment/prognosis. The number of topics covered by each video was recorded as some videos addressed multiple topics. The topic that occupied the longest duration within the video was designated as the main topic. Videos that did not cover these predefined topics were considered irrelevant and excluded from the analysis. The video styles were classified as solo narration, questions and answers (Q&A), presentations (PPT/class), animation/action, medical scenarios, TV shows/documentaries, and other styles. For a more thorough methodology, please refer to Additional File 1.

For video quality assessment, two investigators independently reviewed and scored the videos. In cases of discrepancies or disagreements, a third investigator was consulted to reach a consensus. The quality of the videos was evaluated using the following tools: the Patient Education Materials Assessment Tool (PEMAT), which includes PEMAT-Actionability and PEMAT-Understandability; the Video Information and Quality Index (VIQI); the Global Quality Score (GQS); and the modified DISCERN (mDISCERN). Previous studies have validated these tools, particularly for evaluating health information on social media platforms9,10,11. Further details are provided in Additional Files 2 and 3.

Statistical analysis

Data analysis was performed using IBM SPSS version 24.0. The Shapiro-Wilk test was used to evaluate the normality of continuous variables, with normally distributed variables presented as mean ± SD (standard deviation). Non-normally distributed data were reported as median and interquartile range (M[P25-P75]) and compared using the Mann-Whitney U test. Categorical variables were summarized as frequencies and percentages and compared using the Chi-square test, continuity correction, or Fisher’s exact test as appropriate. Spearman correlation analysis was conducted to assess the relationship between video quality and audience engagement metrics. Statistical significance was defined as P < 0.05.

Results

Video characteristics

After applying the inclusion and exclusion criteria, this analysis ultimately included 300 oral cancer-related videos from YouTube and Bilibili. Based on the results of the Shapiro-Wilk test, all the continuous variables were not normally distributed. Therefore, the Mann-Whitney U test was used for comparisons.

YouTube videos were generally longer, released earlier, and had more views and likes compared to Bilibili videos. However, there were no significant differences in the average likes or comments per 30 days or the total number of comments between the two platforms (Table 1).

Author characteristics

The present study examined a total of 111 uploaders on YouTube and 134 on Bilibili. YouTube authors posted a greater number of videos and owned more subscribers. A higher proportion of YouTube content creators were affiliated with hospitals/non-profit organizations and for-profit companies, while Bilibili’s content was predominantly produced by self-media and doctors. YouTube authors included a greater proportion of professional accounts and group accounts, which also exhibited a higher certification rate (Table 2; Fig. 2A and B).

Video categorization

Videos on both platforms have promoted scientific information about oral cancer to the public to some extent. YouTube videos covered a broader range of topics, with treatment and prognosis being the most prevalent. A large number of YouTube videos introduced surgical treatment for oral cancer, often complemented by radiotherapy and systemic therapy. Early treatment can achieve a favorable prognosis, however, most videos did not provide precise values, such as a five-year survival rate. In contrast, Bilibili videos focused more on etiology and prevention. Many Bilibili videos revealed that betel nut is deemed as the most frequently mentioned cause of oral cancer, reflecting its popularity among the population in southern China. Smoking, HPV (Human Papillomavirus), and alcohol were also mentioned as the common causes of oral cancer. This information helps enhance public awareness and prevention efforts.

Regarding video production styles, solo narration was the predominant format on both platforms. YouTube featured a greater variety of formats, including TV shows and documentaries (Table 3; Fig. 2C and D).

Video quality

YouTube videos scored higher on several quality metrics, including PEMAT-U, VIQI-sum, and GQS. However, there were no significant differences in PEMAT-T, PEMAT-A, and mDISCERN scores between the two platforms (Table 4). It is worth noting that the scores for mDISCERN-4 and mDISCERN-5 were relatively low on both platforms, suggesting that content creators have paid insufficient attention to providing additional references for patient consideration and have not clearly addressed areas of uncertainty.

Videos produced by the professionals were generally of higher quality compared to those by the non-professionals (Table 5). This is specifically reflected in their greater understandability (PEMAT-U), more reliable and more accurate information sources (mDISCERN-2 and VIQI-2), richer and more detailed content(VIQI-3 and VIQI-4), as well as clearer objectives (mDISCERN-1). Some videos created by non-professional authors contained inaccurate information, such as incorrectly attributing oral cancer to candy consumption and excessively exaggerating the efficacy of a specific treatment method.

Correlation analysis

There was no strong correlation between video quality and audience interaction, indicating that viewers had difficulty recognizing high-quality content (Table 6). In general, VIQI exhibited a weak to moderate positive relationship with most aspects of audience interaction across both platforms, while other assessment tools exhibited either no relationship or a weak relationship.

Table 1 Characteristics of videos about oral carcinoma on YouTube and bilibili.
Full size table
Table 2 Characteristics of video uploaders about oral carcinoma on YouTube and bilibili.
Full size table
Fig. 2
figure 2

Characteristics of authors and content on YouTube and Bilibili videos. (A) The number of author types. (B) The number of certified author types. (C) The percentage of topics. (D) The percentage of each video style.

Full size image
Table 3 Categorization of videos about oral carcinoma on YouTube and bilibili.
Full size table
Table 4 Quality assessment of videos about oral carcinoma on YouTube and Bilibili.
Full size table
Table 5 Quality comparison between the videos uploaded by professionals and non-professionals.
Full size table
Table 6 Spearman correlation between video quality and audience interaction on YouTube and bilibili.
Full size table

Discussion

The use of social media for public health education has grown significantly, with digital video playing an important role in overcoming traditional barriers to information access. Short video platforms such as TikTok have gained global popularity; however, they may not match the quality of long-form content12,13. Due to restricted access to YouTube in China and the prominence of Bilibili as a leading Chinese long-form video platform, we selected it as a viable alternative for our study. In the field of oncology, social media content has been investigated for its educational value on various cancers, including nasopharyngeal14, thyroid10,15, laryngeal9, liver11, lung16, and breast cancer17, et al. However, concerns about misinformation remain. For example, in the HINTS (Health Information National Trends Survey) data collected from March through November 2022, approximately 28% and 36% of HINTS respondents (n = 6231) reported that a lot and some of the health information they see on social media was false or misleading, respectively18,19.

Several studies have investigated oral cancer20,21,22,23. Ulloa-Morales noted that online audio-visual material on oral cancer in Spanish is incomplete and of limited usefulness20. This finding is reflected in research on YouTube videos in Portuguese21 and English22,23. However, to the best of our knowledge, no studies have evaluated the quality of oral cancer audio-visual resources in Chinese. Furthermore, considering the rapid growth of social media platforms, these findings may have varied over time.

This study is the first comprehensive evaluation of oral cancer-related videos in English and Chinese across two major long-video platforms. We utilized four assessment tools to conduct a thorough evaluation, revealing that YouTube videos generally performed better but still required improvement. Factors such as originality, certification, and video production style, often overlooked in previous studies, were also taken into account. Our findings not only inform public access to health information but also provide useful insights for content creators and platform developers.

In terms of video characteristics, YouTube was established nearly 20 years ago24, allowing it to accumulate a wealth of audience interaction data. In comparison, Bilibili was established 14 years old11. YouTube’s global reach and multilingual support contribute to its having over 2 billion monthly active users24,25, which also explains the higher number of views and likes observed in our research. However, the lack of significant differences in average likes per 30 days and comments suggests that Bilibili emphasizes on enhancing user experience and encouraging feedback to creators26,27.

With respect to the author characteristics, team accounts comprised a larger proportion of the authors on YouTube, consistent with the findings of previous studies on other diseases9,28. The division of labor within teams enables higher-quality content, more frequent uploads, and the development of larger subscriber bases. In contrast, personal accounts are constrained by limited resources and time, which can negatively affect video quality. Notably, Bilibili featured more doctor-authored content, reflecting the enthusiasm of Chinese doctors for public education. Author certification can enhance video exposure and improve audience engagement according to the platform policy. However, certification rates were unsatisfactory on both platforms, consistent with prior research9. It is recommended that recommend that all professionals pursue certification to improve content credibility and visibility.

The present study revealed variations in video topics and production styles. The divergence in primary topics between the platforms may reflect cultural differences. YouTube’s global audience predominantly seeks cutting-edge therapeutic information, whereas the public in China shows a greater interest in disease prevention. Few studies have investigated video production styles9. Solo narration, due to its simplicity, is the most prevalent style across platforms, particularly for individual creators such as doctors who have limited time and resources. However, it should be noted that solo narration conveys a comparatively limited amount of information in comparison to other styles. The higher prevalence of team accounts on YouTube enables a greater variety of production styles, including TV shows and documentaries, which require meticulous planning and editing. Selecting an optimal video style that balances quality, conciseness, and clarity remains a challenge for content creators.

Tools are essential for video assessment. The most comprehensive tool in this study is the PEMAT tool, which consists of 17 questions and focuses on understandability and actionability. Developed by Shoemaker in 2014, it demonstrates strong internal consistency, reliability, and construct validity29,30. However, its complexity and time-consuming nature may result in inflated scores due to the “non-applicable (NA)” items. The GQS tool, introduced by Bernard in 2007, is simpler and more widely used, assessing both video quality and audience engagement31. Nagpal’s VIQI tool, developed in 2015, complements the GQS by adding a more detailed component rating system32. The DISCERN tool, which was originally designed for textual health information33 but not specifically for video material34, was adapted by Singh in 2012 to assess the reliability of video materials35. All these tools have been widely applied in evaluating the quality of health-related videos.

Only three studies have examined the differences in video quality between YouTube and Bilibili in the field of health-related videos. YouTube videos on laryngeal cancer9 and probiotics36 generally demonstrated higher scores than those on Bilibili, contrasting with Wang’s findings on gastric cancer28. Our results align with the former observations, likely due to factors such as team-based content creation, YouTube’s algorithmic promotion of professional videos, and auxiliary tools for YouTube like vidIQ and Nox Influencer37,38, that enhance video production. Despite these advantages, there is still room for improvement in video quality across all the platforms. We encourage Chinese content creators to translate high-quality YouTube videos and upload them to Bilibili, provided that copyright permissions are respected. Additionally, original high-quality content is also appreciated.

Our research indicated that videos produced by professionals were generally of higher quality. This is consistent with a previous relevant systematic review of social media which showed that misleading information is mostly created by individuals with no official or institutional affiliations39,40. Therefore, there is a need for more professionals to engage in public education to improve the overall quality of oral cancer videos.

The relationship between video quality and audience interaction was weak, consistent with the findings of previous studies, suggesting that viewers may have difficulty assessing the quality of health-related videos9,11,41. Consequently, public health education needs to be strengthened. Platform developers should optimize their algorithms to promote high-quality content based on accuracy and reliability, rather than solely focusing on views and likes. Furthermore, as most platforms tend to allocate more traffic to certified accounts9, we recommend that all professional creators pursue certification to ensure the broader dissemination of professional content. These approaches are crucial for ensuring that viewers receive accurate and reliable health information.

There are several limitations in this study. First, despite the use of four tools and the involvement of three well-trained doctors in the rating process, potential biases may still exist. Secondly, due to platform restrictions, we were unable to access YouTube data on collections and shares, and no platform provides a thumbs-down button for expressing negative opinions. Thirdly, the algorithmic influence of each platform, the influence of engagement metrics on video dissemination, the lack of an advanced search function, and the limited sample size may introduce bias into our results. Fourthly, our findings may not be fully applicable in different linguistic or cultural contexts, and potential cultural biases in content production may exist. Fifthly, additional text information, such as comments, was not analyzed, which may have led to some negative sentiment being overlooked. Sixthly, we did not conduct an in-depth analysis of audience engagement, video quality, and author/content types due to word limitations. Finally, this study represents a snapshot in time from a particular region with a limited sample size. Given the rapid growth of social media, these findings may change over time. While some limitations cannot be avoided, further research should refine methodology by increasing sample size, involving more platforms, analyzing the comment section, and conducting an in-depth analysis of content quality. The use of artificial intelligence techniques like PCA, t-SNE, and UMAP should be considered for further data analysis42.

In conclusion, although YouTube videos generally receive more audience interaction and demonstrate higher video quality compared to Bilibili videos, there is a need for more professional content creators to further improve video quality. For video creators, we recommend greater involvement of professionals and teamwork, as certification and diverse production styles may enhance audience engagement. Additionally, platforms should optimize their algorithms (for example, by offering more support to certified professional authors) to promote high-quality original health content to the public.