ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
Original article 
Real-world Evidence of Implementing eHealth 
Enables Fluent Symptom-based Follow-up of a 
Growing Number of Patients with Breast Cancer 
with the Same Healthcare Resources 
Maarit Bärlund, 1 , 2 Laura Takala, 2 Leena Tianen, 1 , 2 
Pirkko-Liisa Kellokumpu-Lehtinen 2 , 3 
Abstract 
We assessed the impact and added value of two digital tools for healthcare professionals in monitoring 1420 
patients with breast cancer in the follow-up phase. Accessibility was improved by 55%, and three times more 
patient contacts could be treated by the same number of nurses as before. Two nurse feedback surveys favored 
the use of the digital tools. 
Background: The growing number of new breast cancer (BC) patients has indicated an urgent need for better tools to 
monitor patients. Due to the increasing number of unscheduled patient calls, two digital applications were implemented: 
a callback application (CA) and a digital cancer follow-up application (CFUA). The aim of this study was to evaluate the 
implementation and added value of digital applications for healthcare professionals in monitoring BC patients from 2012 
to 2020. Materials and Methods: As of the end of 2020, 1420 follow-up BC patients were enrolled in the CFUA, which 
covered 70% of the BC patients in the follow-up phase in the clinic. All unscheduled telephone calls and digital contacts 
were recorded from January 2015 to December 2020. Results: The implementation of the CA revealed low telephone 
accessibility (56%), which was improved by 55% by the end of 2020 due to the implementation of the CFUA. In 2019, 
59% of all phone service contacts were digitally prepared, and contact reasons were classified by the patients. One of 
the most important findings was that BC patient contacts more than tripled during the nine-year study period, and these 
patients could be treated by the same number of nurses due to the implementation of the CA and CFUA. Feedback 
from two nurses’ surveys favored the use of the CFUA. Conclusion: Our study clearly demonstrates that new digital 
tools are useful in the challenge of a growing number of BC patients in surveillance who should be handled with limited 
health care resources. 
Clinical Breast Cancer, Vol. 000, No.xxx, 1–8 © 2021 Elsevier Inc. All rights reserved. 
Keywords: Patient-reported outcomes, PROs, Real-world data, Digital application, Breast Cancer 
 
 
A
 
 
 Introduction 
Breast cancer (BC) is the most common type of cancer in
women. 1 The number of BC patients and survivors has increased,Abbreviations: BC, breast cancer; PRO, patient-reported outcome; CA, a callback appli- 
cation; CFUA, a digital cancer follow-up application; CTCAE, common terminology 
criteria for adverse events; VAS, visual Analogy Scale. 
1 Department of Oncology, Tays Cancer Center, Tampere University Hospital and 
Faculty of Medicine and Health Technology, Tampere, FI-33521 Tampere, Finland 
2 Departrment of Medicine and Health Technology, University of Tampere, Tampere, 
Finland, FI-33101 University of Tampere 
3 Development and Innovation Center, Tampere University Hospital, Tampere, 
Finland, FI-33521 Tampere, Finland 
Submitted: May 2, 2021; Revised: Sep 11, 2021; Accepted: Sep 15, 2021; Epub: xxx 
ddress for correspondence: Maarit Bärlund, MD, PhD, MSc, Department of Oncol- 
ogy, Tays Cancer Center, Tampere University Hospital, PO Box 2000, FI-33521 
Tampere, Finland. 
E-mail contact: maarit.barlund@pshp.fi
 
 
 
 
 
 
 
 
 
 
 
1526-8209/$ - see front matter © 2021 Elsevier Inc. All rights reserved. 
https://doi.org/10.1016/j.clbc.2021.09.005 
Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Cliespecially in the Western world, as most patients can be cured
due to earlier diagnostics and more effective treatments. Thus, a
new challenge is how to best support patients after BC treatments
with limited health care resources. 2 , 3 Many patients experience a
wide range of ongoing symptoms. 4 Unmanaged symptoms and
side effects can lead to worsening of health-related quality of life
and clinical outcomes among BC patients. 4 , 5 In a recent article
by Absolom et al. , real-time monitoring with an electronic patient-
reported outcome (PRO) system improved the physical well-being
of these cancer patients. 6 
In many countries, conventional follow-up strategies, such as
routine examination by a specialist, have been replaced with less
intensive approaches. These substitutions have not affected the main
endpoint, overall survival. 2 A nurse-led follow-up schedule seems to
be an appropriate way to reduce routine clinic visits, including costs,Clinical Breast Cancer 2021 1 
enting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
nical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Real-world Evidence of Implementing eHealth Enables Fluent Symptom-based 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2 Cliand represents an accepted alternative strategy in real-world clinical
practice. 7 , 8 
In 2007, at the Department of Oncology, Tampere Univer-
sity Hospital, Finland, regular appointments during five years of
BC follow-up were reduced to four appointments. Additionally, a
BC nurse-led phone service was set up for patients who needed
counselling about symptoms related to possible side effects or recur-
rence. The BC nurse coordinated the follow-up visits, including
regular mammography examinations, and consulted a BC oncolo-
gist when needed. 
Over the last few years, the number of unscheduled patient calls
to phone services has consistently increased, and the need for new
caregivers has arisen. Therefore, two digital applications (Aurora
TeleQ callback application (CA) 9 and Noona 10 , a cancer follow-
up application (CFUA)) were introduced to improve BC patient
services and to avoid the need to increase the number of caregivers.
The objective of this study was to evaluate the implementation and
the added value of digital applications for healthcare professionals
in monitoring BC patients during a nine-year period from 2012 to
2020. 
Materials and Methods 
The BC nurse-led telephone service was organized as follows.
1) The telephone service was available four days of a week with
limited open hours. 2) BC nurses were trained to answer the most
common questions independently and to direct abnormal situations
for assessment by the BC oncologists. 3) The BC nurse was able to
easily consult a BC oncologist. 
Aurora TeleQ callback application (CA) (Aurora Innovation Oy,
Uppsala, Sweden) is a web-based queue-handling application that
does not require a switchboard. The CA receives unscheduled calls,
asks patients to enter their phone number, and then tells the patient
when callback contact will take place. The CA estimates a sched-
uled callback time based on the number of incoming calls and their
average length. The application was implemented by redirecting the
telephone number of the phone service to the CA at the beginning
of January 2015. An analytical tool integrated into the CA provides
statistical reports. 
Noona (Varian Medical Systems, Inc. Palo Alto, CA), a modular
digital CFUA, was implemented into a clinical routine for BC
patients at our university hospital in October 2016. BC patients
were offered the opportunity to start using the CFUA as a volun-
tary and complimentary communication method for BC surveil-
lance after primary treatment. The implementation of the CFUA
was initiated by mass recruitment, and utilizing the functionality
of the CFUA was subsequently continued in face-to-face clinical
appointments. 
Patients can use the CFUA to report their symptoms in a struc-
tured online format and submit the report to the clinic. The
online format includes the most relevant symptom-related details.
A symptom library was formed from the most common symptoms
during the BC follow-up phase: one-sided swelling of an arm
or leg, pain, changes in the breast or operation area, nodules
and lumps, joint symptoms, changes in mood and emotions,
menopausal symptoms, stomach and bowel symptoms and respi-
ratory symptoms. Patients were asked if they had oedema, and ifnical Breast Cancer 2021 
Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Clso, whether it was on the operated or non-operated side. Pain was
measured with the Visual Analogy Scale (VAS) from 0 to 10 (0 = no
pain to 10 = severe pain). 11 In addition, the patient reported the
pain location by drawing it on a human figure in the CFUA. Side
effects were assessed with the Common Terminology Criteria for
Adverse Events (CTCAE 4.0 grading system). 12 
The CFUA automatically analyses the urgency of the contact and
directs the patient to the most appropriate point of care. In addition,
patients can contact the clinic regarding topics other than symptoms
via an open question form with one of six message topics (medica-
tion, follow-up period, treatment, treatment side effects, physio-
therapy, and other concerns). In addition, patients are advised to
attach photos, for example, of skin changes. For the clinic, the main
CFUA functionality is a work queue for new patient contacts and
for communicating directly with patients. 
The caregivers who use this CFUA on a daily basis include BC
nurses and oncologists in our clinic. Healthcare professionals can
contact a patient on demand either through the CFUA or by phone.
Additionally, the nurse can consult an oncologist via the CFUA,
which is a stand-alone software platform and is not interfaced with
electronic patient records. 
The number of unscheduled patient calls to the BC nurse-led
telephone service was recorded manually by nurses during 2012-
2014. The analytics tool of the CA registered the number and
duration of calls outside office hours and how many times the
caregiver had to call the patient before reaching the patient from
2016 onwards. The analytics tool of the CFUA registered the
number and age of patient users, the number of digital contacts,
the number of symptom contacts out of all contacts, and the reason
for the contact by classification. 
Nurses completed online feedback surveys after seven months of
CFUA experience in November 2017 and after five years of use in
February 2021. 
Results 
The implementation of the CA revealed low telephone helpline
accessibility (56%) ( Table 1 ). This was mainly due to patient calls
outside of operating hours. According to the CA, the average
telephone call duration was 9 minutes. For 25% of calls, the nurse
had to call twice or more to contact the patient, although the patient
was supposed to be awaiting a return call. 
The digital invitation to the CFUA was sent to 1000 BC patients
through a mass recruitment utilizing the functionality of the CFUA.
As a result of the recruitment, 58% of invited patients accessed
the CFUA. Thereafter, recruitment was continued at face-to-face
appointments, which was a notably slower recruitment method
( Figure 1 ). In 2019, the number of patient recruitments decreased.
At the end of 2020, 1420 patients had enabled the use of CFUA,
which accounted for 70% of all BC patients in the follow-up phase
in our clinic. The mean age of CFUA users was 64 years (range
32-85 years). However, only half of the patients who enabled the
application used the CFUA to contact the clinic, and the number
of contacts from individual patients was moderate, with a maximal
number ranging from 3 to 13 ( Table 2 ). 
In 2016, digital contacts accounted for only 9.1% of all contacts.
However, digital contacts increased rapidly: 43.7% of all phoneenting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Maarit Bärlund et al 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
Table 1 Number of Calls and Digital Contacts Presented on an Annual Basis 
Year Phone contacts (%) Digital contacts (%) Answered contacts Unanswered calls Accessibility (%) 
2012 863 (100.0) 0 863 n.a ∗. n.a ∗. 
2013 1361 (100.0) 0 1361 n.a ∗. n.a ∗. 
2014 1730 (100.0) 0 1730 n.a ∗. n.a ∗. 
2015 1707 (100.0) 0 1707 1322 56 
2016 1832 (90.9) 184 (9.1) 2016 1240 62 
2017 1232 (56.3) 957 (43.7) 2189 844 72 
2018 1140 (47.1) 1280 (52.9) 2420 1021 70 
2019 1074 (40.1) 1607 (59.9) 2681 584 82 
2020 2223 (54.0) 1905 (46.0) 4128 596 87 
Answered Contacts Include Both Calls and Digital Contacts. Unanswered calls include the number of patients calls outside operating hours. Accessibility rate is the percentage of all contacts who 
responded either digitally or by telephone. 
n.a. = not available. 
Table 2 The Number of CFUA Deployers per Year was Recorded at the End of Each Year 
Year Number of CFUA 
deployers 
Number of individual patients 
contacting the clinic 
Maximal number of 
digital contacts/patient 
2016 657 62 3 
2017 830 473 8 
2018 1012 559 8 
2019 1085 573 9 
2020 1468 703 13 
Only half of those who deployed the CFUA contacted the clinic by the CFUA. The maximal number of digital contacts from individual patients was moderate. 
Figure 1 The cumulative number of patients with breast cancer in the follow-up phase who adopted the CFUA is presented 
monthly. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 service contacts were digital in 2017 and 59% in 2019. Telephone
calls decreased as digital contacts increased; the decrease from 2016
to 2019 in the number of telephone calls was 41% (n = 758)
( Figure 2 ). The phone service accessibility rate increased to 87%
by the end of 2020, which was improved by 55% compared to the
end of 2015 due to the CFUA ( Table 1 ). In 2015, the BC nurses
spent 2858 hours on the phone, but with the implementation of
the CFUA, this time began to decline to 1898 hours in 2018 and to
2080 hours in 2019. The COVID-19 pandemic had an impact on
unscheduled contacts during the spring of 2020; the number of callsPlease cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Cland time spent on the phone by BC nurses doubled ( Figures 2 and
3 ). BC patient unscheduled contacts more than tripled during the
nine-year study period. Another important finding was that 59%
of all phone service contacts were digitally prepared, and contact
reasons were classified by the patients using the CFUA in 2019. 
Patients had previously completed the questions in the CFUA;
thus, the digital contacts already had symptoms prepared and
other reasons clarified. One-third of digital contacts were symptom-
related contacts, and the percentage of symptom-related contacts
remained stable when the number of contacts increased accordingClinical Breast Cancer 2021 3 
enting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Real-world Evidence of Implementing eHealth Enables Fluent Symptom-based 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
Figure 2 Annual number of unscheduled calls and digital contacts. Telephone calls began to decline after the CFUA was 
introduced. The number of contacts in 2020 increased due to the COVID-19 pandemic, with a doubling of telephone 
calls. 
Figure 3 Changes in the number of calls and digital contacts presented on a monthly basis. The impact of the COVID-19 
pandemic on the number of unscheduled contacts to the clinic is reflected in the spring 2020 contact numbers. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4 Clito CFUA statistics ( Figure 4 ). The three most common reasons
for digital symptom-related contacts were pain, one-sided swelling
of an arm, and changes in the breasts or operation area. Other
common reasons for non-symptom-related contacts were the follow-
up period, physiotherapy, medication and treatment-related side
effects. 
The CFUA enabled communication with BC patients in a new
way: a message from the clinic could be sent to all BC patients
with one click, and this functionality was utilized annually. For
example, bulletins related to influenza vaccinations and the effect
of the COVID-19 pandemic on BC follow-up visits proved to be a
valuable way of reaching patients with urgent information. 
After one year of CFUA experience, the first nurse feedback
survey was conducted in November 2017. The survey was
completed by 19 nurses, 59% of whom were over 40 years of age.
All were experienced smartphone and internet users. Only 3 nurses
(16%) had no experience using social media, and 2 (10%) had no
experience using tablets. All nurses acknowledged that the clinic hadnical Breast Cancer 2021 
Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Clclear instructions for deploying the CFUA. The CFUA implemen-
tation was easy for 18 nurses (95%), and a majority (83%) used
the CFUA more than five times. Additionally, 61% of the nurses
had contacted patients using the CFUA. Compared to the phone,
only 5 nurses (27%) stated that communicating with patients using
the CFUA was difficult. Fourteen nurses (74%) found the CFUA
practical. The majority of the nurses (N = 16, 85%) preferred using
both the phone and the CFUA for patient communication, and only
3 (16%) preferred to use the CFUA alone. 
The second nurse feedback survey was performed five years after
the implementation of the CFUA. The survey was completed by a
total of 20 nurses, half of whom were over 40 years of age. Most
nurses (70%) had more than two years of experience using the
CFUA, and 19 nurses (95%) had communicated with patients via
the CFUA. Furthermore, all nurses felt that the instructions for
using the CFUA were clear. Most nurses (80%) used the CFUA
daily, and the rest (20%) used the CFUA weekly. Figure 5 presents
the results of the nurse survey. In terms of clinical utility, 18 nursesenting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Maarit Bärlund et al 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
Figure 4 Digital contacts are presented monthly. All digital contacts are presented in blue, and symptom-related contacts are 
presented in red. One-third of digital contacts were symptom-related throughout the study period. 
Figure 5 Summary of the second nurse feedback survey responses. 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 (95%) indicated that patient self-reported information was helpful
for documentation into the electronic medical record, 17 (85%)
found that the CFUA was more flexible and faster than a phone
call, and 19 (95%) felt that patient self-reported information was
useful for patient care. In terms of quality and value of care, 17
nurses (85%) believed that the system improved the quality of
care for patients, 19 (95%) would use the CFUA in the future,
and 18 (90%) would recommend it to other clinics. Most nurses
(75%) preferred using both the phone and the CFUA for patient
communication, only 1 (5%) preferred the phone only, and 4 (20%)
preferred the CFUA only. In addition, most of the nurses answering
were same in both questionnaires. 
Discussion 
We assessed the impact and value of two digital tools (the CA
and CFUA) in monitoring the follow-up period of BC patients by
health care professionals, mainly nurses. One of the most impor-Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Cltant findings was that BC patient contacts more than tripled during
the nine-year study period, and these contacts could be treated by
the same number of nurses due to the implementation of the CA
and CFUA. Two nurse feedback surveys demonstrated favorable
perceptions of the clinical utility and impact on the quality and
value of care of the CA and CFUA. After four years of use, 59% of
the unscheduled contacts were digitally prepared, and other contact
reasons were clarified ahead of time. 
When digital applications are implemented, it is important to
achieve the anticipated benefits of their features and functions in
clinical logistics. 13 The implementation of the CA was straightfor-
ward and did not require a designed implementation plan. However,
for the implementation of the CFUA, the mass recruitment of
patient users was considered a crucial step for successful implemen-
tation. This approach enabled the immediate full utilization of the
CFUA and the accumulation of positive experiences by nurses. Face-
to-face recruitment depends on each employee’s motivation and
activity. The method is slow, and using an iterative approach toClinical Breast Cancer 2021 5 
enting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Real-world Evidence of Implementing eHealth Enables Fluent Symptom-based 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6 Clithe implementation could have made personal recruitment more
efficient. 14 In the future, intelligent robotic process automation
could solve patient-user recruitment for the CFUA. 15 Another
significant factor for success was the instructions for the use of the
CFUA for both patients and BC nurses. The second survey demon-
strated that the use of the CFUA had become routine in the clinic,
as all nurses used it daily or at least weekly. 
The implementation of the CA made it possible to monitor the
accessibility rate of telephone services, which was far too low in
2015 (56%). Introducing the CFUA improved the accessibility rate
of telephone services by 55%. For patient users of the CFUA, the
accessibility rate is always 100%. Telephone triage services provided
by BC nurses enhance necessary supportive care for patients who
experience symptoms during the BC follow-up period. The CA
enabled monitoring of unscheduled calls, including the duration
of calls and missed calls. At the same time, the CFUA enabled
an inexpensive structured format for data collection from patients
about their symptoms, such as the time series, location, and inten-
sity of symptoms. Exact data about patient calls and symptoms
enabled the quality assessment of telephone triage and symptom
management and the optimization of continuous care. Outcomes of
telephone triage studies include improved cancer symptoms, prompt
access to specialist care, reduced visits to the emergency room (ER),
fewer hospital admissions, reduced costs, and telephone calls about
symptoms from a nurse, which were able to solve the problem
without a physician in 87% of calls. 16-18 Information on the quality
of telephone triage services for patients with cancer is limited. 16
Only one study showed a need to improve the quality of cancer
symptom management. 19 This study has three main limitations.
The first limitation is the lack of clinical outcome of patients with
breast cancer, which is not necessary, since the main purpose of
this study was the evaluate usefulness of these digital tools among
patients with breast cancer and nurses treating them. The second
limitation is the lack of patient questionnaires. However, we have
published data of patient reported outcome in our previous study
which included partly same patient populations with breast cancer. 27
The last limitation of our study includes patients’ self-classification
of their contacts, and a lack of caregiver evaluation of the accuracy
of patients’ classifications. 
The CA eliminated nurses’ work interruptions by muting the
phone. Surprisingly, in a quarter of the CA calls, nurses had to call
several times to contact the patient, although the patient was waiting
for a return back. This may be due to the nurse calling the patient
at an inconvenient time. The CFUA allows the patient to read the
message at a time that is appropriate for them. Telephone services
often create communication bottlenecks in healthcare services. With
the introduction of the CFUA, the number of incoming calls to
the telephone service dropped by 41%. The CFUA enabled BC
nurses to reduce the frequency of unscheduled patient calls and time
spent on the phone and decreased unnecessary service contacts, eg,
telephone calls that the patient did not answer. In addition, CFUA
templates for the most common reasons for contacting the clinic
can reduce the amount of time spent on each case. The CFUA
prioritizes patients’ digital contact by symptom severity and urgency,
provides a work queue and maps patients’ symptoms structurally tonical Breast Cancer 2021 
Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Clthe caregiver. An optimized workflow increases independence in the
work of orderlies. 
In summary, the CFUA can help to redefine and reform how
patient flow is managed. This means that the same number of BC
nurses can manage more contacts in a shorter period, saving time for
direct patient work at the clinic. According to this study, BC nurses
handled more than three times as many patient contacts using these
new digital tools. 
In a retrospective study that assessed the follow-up of 303 patients
with localized BC, the first indicators of advanced BC recurrence
were symptoms for 47.2% of patients. 20 In our study, one-third
of digital contacts were symptom-related contacts, and the most
common symptoms were pain, one-sided swelling of an arm, and
changes in the breast and operation area. Our findings are similar
to those of previous studies. 21-23 Further studies to explore patient
symptom contacts are ongoing. 
Approximately two-thirds of early-stage BC is hormone receptor
positive and treated with 5- to 10-year adjuvant endocrine therapy
(AET). AET decreases BC mortality, improves 15-year survival by
one-third, and reduces the risk of BC recurrence by up to 50%
24 . Despite its clinical benefits, adherence to AET is poor, with
up to 59% of patients not taking medication as recommended 25 .
Adherence is the single most important factor that influences treat-
ment outcomes. Preliminary results of a virtual intervention for
adherence, symptom management, and distress seem promising. 26
In our recent study, patients were committed to using the CFUA,
with PRO questionary response rates as high as 89%, 27 which
may be because CFUA enables two-way communication between
caregivers and patients. Combining virtual intervention for adher-
ence, symptom management, distress, and healthy lifestyles with the
CFUA could improve patients’ quality of life and BC treatment
outcomes as shown in previous studies. 28 , 29 In our future studies
we will evaluate the aspects of adherence in our patient populations
with breast cancer and their treating nurses. Recent expert reports
estimate that successful lifestyle changes could prevent 25% to 30%
of breast cancer cases. 30 
In 2015, a Finnish study was launched with the goal of identi-
fying outpatient preferences for the modality of follow-up. 31 A
total of 765 patients with early BC were randomized at the end
of adjuvant radiotherapy to follow-up by mobile Noona software
or phone calls. After six months, the groups were crossed over to
the other arm. At 12 months, the patients were asked about their
preferences. Thirty percent of the patients preferred Noona and
30% preferred the phone, while 30% considered both modalities
equally good. Very similar results were seen in a large ongoing PRO
implementation study in the USA, in which approximately 40% of
patients chose to use an automated telephone system rather than an
electrical PRO system. 32 According to the patient-oriented health-
care service design, patients must be offered digital and traditional
telephone services. 33 Therefore, according to this study, the combi-
nation of the CA and CFUA is the optimal technical solution for BC
patients’ service support and from a healthcare resourcing perspec-
tive. Actively supporting the use of online services can ensure phone
lines for those who either do not have access or are unable to use
online solutions. Our study demonstrated that age was not a barrierenting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Maarit Bärlund et al 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 to the use of digital services. The average age of patient users was 64
years, and the oldest users were over 80 years of age. 
Conclusions 
Our study is the first real-world study to evaluate the PRO
system’s clinical impact for monitoring BC patients in the follow-
up phase with 5 years of experience. New digital tools can help
with the growing number of BC survivors with limited health care
resources. A prerequisite for fully utilizing digital services is careful
planning of the implementation and service design utilization. 34 
Ideally, the CFUA would be more fully integrated with the EPR to
enable seamless information flow of patient-reported information
into the EPR for provider viewing. There is an enormous amount of
potential that needs to be harnessed and incorporated into the BC
patient care pathway. According to recent studies, the patients who
derived the most benefit from telemedicine were those who had a
BC diagnosis. 35-38 
Clinical Practice Points 
A new challenge is how to best support growing numbers of BC
survivors with limited health care resources. Digital health solutions
offer an innovative approach to optimize healthcare resource utiliza-
tion. Although the benefits of digital health solutions have been
confirmed in the setting of randomized clinical trials, there is a need
to continue to evaluate their efficacy and efficiency in real-world
conditions. In 2007, at the Department of Oncology, Tampere
University Hospital, Finland, a BC nurse-led follow-up and phone
service was set up for patients who needed counselling about
symptoms related to possible side effects or recurrence. Over the last
few years, the number of unscheduled patient calls to phone services
has increased consistently, and new caregivers’ needs have arisen. 
Our study shows that the optimal implementation of two digital
applications, the CA and the digital CFUA, was clearly helpful for
healthcare professionals in monitoring 1420 patients with breast
cancer in a 5-year follow-up phase. 
The accessibility of nurse-led telephone services was improved by
55%, and three times more patient contacts could be treated by
the same number of nurses as before. Two nurse feedback surveys
favored the use of the CFUA. This is one of the first studies to
present an evaluation in a real-world setting. 
Supplemental Figures 1 and 2. Screenshot of the patients’ side of
the cancer follow-up application (CFUA), which allowed patients
to contact the clinic. The CFUA offered symptom or non-symptom
contact options. The most important symptoms related to the breast
cancer follow-up period were symptom contact options: one-sided
swelling of an arm or leg, pain, changes in the breasts or surgical site,
nodules and bumps, respiratory symptoms and other symptoms.
Non-symptom contact options were medication, follow-up period,
treatment, treatment side effects, physiotherapy or other issues.
Images have been provided by Varian Medical Systems. 
Acknowledgments 
This study was financially supported by the Competitive State
Research Financing of the Expert Responsibility Area of Tampere
University Hospital, grant number 9V019 , 9AA027 and 9AB027 .Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, ClThe funder had no role in study design, data collection and analy-
sis, decision to publish, or preparation of the manuscript. The
authors would like to thank breast cancer nurses Titta Leponiemi,
Pia Vänskä, and Päivi Hammar for recording manually unscheduled
telephone calls made to telephone service. The authors would also
like to thank Juha Salonen and Mikko Haltamo for their assistance
with the data collection of the Noona application and digital nurse
Elina Rinkineva for her assistance with the data collection of the
Aurora Callback application. 
Trial Registration 
This study was registered and approved by the Regional Ethics
Committee of Tampere University Hospital in Finland (approval
code R17077). 
Disclosure 
The authors report no conflicts of interest. The Authors did not
have any commercial relationship with the Varian Medical Systems,
Inc company and the company was not involved in planning,
analyzing of interpretation of the results of this study. 
Supplementary materials 
Supplementary material associated with this article can be found,
in the online version, at doi: 10.1016/j.clbc.2021.09.005 . 
References 
1. Ferlay J, Colombet M, Soerjomataram I, et al. Estimating the global cancer
incidence and mortality in 2018: GLOBOCAN sources and methods. Int J Cancer .
2019;144:1941–1953. doi: 10.1002/ijc.31937 . 
2. Moschetti I, Cinquini M, Lambertini M, Levaggi A, Liberati A. Follow-up
strategies for women treated for early breast cancer. Cochrane Database Syst Rev .
2016;2016. doi: 10.1002/14651858.CD001768.pub3 . 
3. Høeg BL, Bidstrup PE, Karlsen RV, et al. Follow-up strategies following comple-
tion of primary cancer treatment in adult cancer survivors. Cochrane Database Syst
Rev . 2019;2019. doi: 10.1002/14651858.CD012425.pub2 . 
4. Denis F, Basch E, Septans AL, et al. Two-year survival comparing web-based
symptom monitoring vs routine surveillance following treatment for lung cancer.
JAMA . 2019;321:306–307. doi: 10.1001/jama.2018.18085 . 
5. Gotay CC, Kawamoto CT, Bottomley A, Efficace F. The prognostic significance
of patient-reported outcomes in cancer clinical trials. J Clin Oncol . 2008;26:1355–
1363. doi: 10.1200/jco.2007.13.3439 . 
6. Absolom K, Warrington L, Hudson E, et al. Phase III randomized controlled
trial of eRAPID: eHealth intervention during chemotherapy. J Clin Oncol .
2021;39:734–747. doi: 10.1200/jco.20.02015 . 
7. Koinberg IL, Fridlund B, Engholm GB, Holmberg L. Nurse-led follow-up on
demand or by a physician after breast cancer surgery: a randomised study. Eur J
Oncol Nurs . 2004;8:109–117. doi: 10.1016/j.ejon.2003.12.005 . 
8. Kimman ML, Bloebaum MM, Dirksen CD, Houben RM, Lambin P, Boersma LJ.
Patient satisfaction with nurse-led telephone follow-up after curative treatment for
breast cancer. BMC Cancer . 2010;10:174. doi: 10.1186/1471- 2407- 10- 174 . 
9. Aurora teleQ-Aurora Innovation. Sweden. More efficient workflow with Aurora
teleQ. https://aurorainnovation.com/en/aurora-teleq/ ; 2021 Accessed February 20,
2021. 
10. Noona. Varian. USA. Log in. https://www.noonaclinic.com/patient/#/login ; 2021
Accessed February 15, 2021. 
11. McCormack HM, de L Horne DJ Sheather S. Clinical applications of visual
analogue scales: a critical review. Psychol Med . 1988;18:1007–1019. doi: 10.1017/
s0033291700009934 . 
12. National Cancer Institute. USA. Common Terminology Criteria for Adverse
Events (CTCAE) | Protocol Development | CTEP. https://ctep.cancer.gov/
protocoldevelopment/electronic _ applications/ctc.htm ; 2021 Accessed February
20, 2021. 
13. Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled
promises of health information technology. Health Aff. 2013;32:63–68. doi: 10.
1377/hlthaff.2012.0693 . 
14. Ross J, Stevenson F, Dack C, et al. Developing an implementation strategy for a
digital health intervention: an example in routine healthcare. BMC Health Serv Res .
2018;18:794. doi: 10.1186/s12913- 018- 3615- 7 . 
15. Davenport T, Kalakota R. The potential for artificial intelligence in healthcare.
Future Healthc J . 2019;6:94–98. doi: 10.7861/futurehosp.6- 2- 94 . Clinical Breast Cancer 2021 7 
enting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005 
Real-world Evidence of Implementing eHealth Enables Fluent Symptom-based 
ARTICLE IN PRESS 
JID: CLBC [mNS;October 13, 2021;21:1 ] 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8 Cli16. Northfield S, Button E, Wyld D, Gavin NC, Nasato G, Yates P. Taking care of
our own: a narrative review of cancer care services-led models of care providing
emergent care to patients with cancer. Eur J Oncol Nurs . 2019;40:85–97. doi: 10.
1016/j.ejon.2019.02.005 . 
17. Suh SR, Lee MK. Effects of nurse-led telephone-based supportive interventions
for patients with cancer: a meta-analysis. Oncol Nurs Forum . 2017;44:E168–E184.
doi: 10.1188/17.onf.e168-e184 . 
18. Remy C, Borniard J, Perez J. Analysis of unscheduled telephone calls received by a
specialized cancer pain nurse. Pain Manag Nurs . 2020;21:255–258. doi: 10.1016/j.
pmn.2019.07.009 . 
19. Stacey D, Jolicoeur L, Balchin K, et al. Quality of telephone nursing services for
adults with cancer and related non-emergent visits to the emergency department.
Can Oncol Nurs J . 2020;30:193–199. doi: 10.5737/23688076303193199 . 
20. Viot J, Bachour M, Meurisse A, Pivot X, Fiteni F. Follow-up of patients with local-
ized breast cancer and first indicators of advanced breast cancer recurrence: a retro-
spective study. Breast . 2017;34:53–57. doi: 10.1016/j.breast.2017.05.005 . 
21. Clough-Gorr KM, Ganz PA, Silliman RA. Older breast cancer survivors: factors
associated with self-reported symptoms of persistent lymphedema over 7 years of
follow-up. Breast J . 2010;16:147–155. doi: 10.1111/j.1524-4741.2009.00878.x . 
22. Leysen L, Beckwée D, Nijs J, Pas R, Bilterys T, Vermeir S, Adriaenssens N. Risk
factors of pain in breast cancer survivors: a systematic review and meta-analysis.
Support Care Cancer . 2017;25:3607–3643. doi: 10.1007/s00520- 017- 3824- 3 . 
23. Montgomery DA, Krupa K, Cooke TG. Locoregional relapse after breast cancer:
most relapses occur late and are not clinically detected. Breast J . 2009;15:163–167.
doi: 10.1111/j.1524-4741.2009.00691.x . 
24. Davies C, Godwin J, et al.Early Breast Cancer Trialists’ Collaborative Group
Relevance of breast cancer hormone receptors and other factors to the efficacy
of adjuvant tamoxifen: patient-level meta-analysis of randomised trials. Lancet
(London, England) . 2011;378:771–784. doi: 10.1016/S0140- 6736(11)60993- 8 . 
25. Makubate B, Donnan PT, Dewar JA, Thompson AM, McCowan C. Cohort study
of adherence to adjuvant endocrine therapy, breast cancer recurrence and mortality.
Br J Cancer . 2013;108:1515–1524. doi: 10.1038/bjc.2013.116 . 
26. Jacobs JM, Walsh EA, Rapoport CS, et al. Development and refinement of
a telehealth intervention for symptom management, distress, and adherence to
adjuvant endocrine therapy after breast cancer. J Clin Psychol Med Settings . 2020.
doi: 10.1007/s10880- 020- 09750- 4 . 
27. Takala L, Kuusinen TE, Skyttä T, Kellokumpu-Lehtinen PL, Bärlund M.
Electronic patient-reported outcomes during breast cancer adjuvant radiotherapy.
Clin Breast Cancer . 2020. doi: 10.1016/j.clbc.2020.10.004 . nical Breast Cancer 2021 
Please cite this article as: Maarit Bärlund et al, Real-world Evidence of Implem
Number of Patients with Breast Cancer with the Same Healthcare Resources, Cl28. Neuhouser ML, Aragaki AK, Prentice RL, et al. Overweight, obesity, and
postmenopausal invasive breast cancer risk: a secondary analysis of the women’s
health initiative randomized clinical trials. JAMA Oncol . 2015;1:611–621. doi: 10.
1001/jamaoncol.2015.1546 . 
29. Block G, Azar KM, Romanelli RJ, et al. Diabetes prevention and weight loss with
a fully automated behavioral intervention by email, web, and mobile phone: a
randomized controlled trial among persons with prediabetes. J Med Internet Res .
2015;17:e240. doi: 10.2196/jmir.4897 . 
30. Britt KL, Cuzick J, Phillips KA. Key steps for effective breast cancer prevention.
Nat Rev Cancer . 2020;20:417–436. doi: 10.1038/s41568- 020- 0266- x . 
31. Mattson J, Peltola M, Poikonen-Saksela P, et al. Follow-up of early breast cancer
(BC) patients by telephone or mobile software. J Clin Oncol . 2018;36:e18883.
doi: 10.1200/jco.2018.36.15 _ suppl.e18883 . 
32. Stover AM , Henson SJJ . Demographic and symptom differences in PROTECT
trial (AFT-39) cancer patient electing to complate weekly home patient-reported
outcome measures (PROMs) via an automated phone call vs. email: implication for
implementing PROs into routine care. Qual Life Res . 2019;28(Suppl 1):S1–S190 . 
33. Santana MJ, Manalili K, Jolley RJ, Zelinsky S, Quan H, Lu M. How to practice
person-centred care: a conceptual framework. Health Expect . 2018;21:429–440.
doi: 10.1111/hex.12640 . 
34. Shaw J, Agarwal P, Desveaux L, et al. Beyond "implementation": digital
health innovation and service design. NPJ Digit Med . 2018;1:48. doi: 10.1038/
s41746- 018- 0059- 8 . 
35. Yildiz F, Oksuzoglu B. Teleoncology or telemedicine for oncology patients during
the COVID-19 pandemic: the new normal for breast cancer survivors? Future
Oncol . 2020;16:2191–2195. doi: 10.2217/fon- 2020- 0714 . 
36. Sonagli M, Cagnacci Neto R, Leite FPM, Makdissi FBA. The use of telemedicine
to maintain breast cancer follow-up and surveillance during the COVID-19
pandemic. J Surg Oncol . 2021;123:371–374. doi: 10.1002/jso.26327 . 
37. Schuster-Bruce AT, Middleton HAR, Macpherson C, Pearce BCS, Evans A. Patient
satisfaction with nurse-led end of treatment telephone consultation for breast
cancer during COVID-19 pandemic. Breast J . 2021;27:77–79. doi: 10.1111/tbj.
14034 . 
38. Zimmerman BS, Seidman D, Berger N, et al. Patient perception of telehealth
services for breast and gynecologic oncology care during the COVID-19 pandemic:
a single center survey-based study. J Breast Cancer . 2020;23:542–552. doi: 10.4048/
jbc.2020.23.e56 . enting eHealth Enables Fluent Symptom-based Follow-up of a Growing 
inical Breast Cancer, https://doi.org/10.1016/j.clbc.2021.09.005