|Ahead of print publication
The Palliative Care Outcome Scale: Turkish validity and Reliability Study
Vildan Kocatepe1, Emel Emine Kayıkçı2, Ülkü Saygılı3, Dilek Yıldırım4, Gülbeyaz Can5, Güngör Örnek6
1 Department of Nursing, Faculty of Health Sciences, Acıbadem Mehmet Ali Aydinlar University, Istanbul, Turkey
2 Department of Nursing, Faculty of Health Sciences, Istanbul Medeniyet University, Istanbul, Turkey
3 Department of Health Care Services, Vocational School of Health Sciences, Selcuk University, Konya, Turkey
4 Department of Nursing, Faculty of Health Sciences, Istanbul Sabahattin Zaim University, Istanbul, Turkey
5 Department of Nursing, Florence Nightingale Nursing Faculty, stanbul University-Cerrahpasa, Istanbul, Turkey
6 Palliative Care Department, Bagcılar Training and Research Hospital, Istanbul
|Date of Submission||02-Jun-2019|
|Date of Acceptance||15-Sep-2019|
|Date of Web Publication||27-Jan-2020|
Department of Nursing, Faculty of Health Sciences, Acıbadem Mehmet Ali Aydınlar University, Istanbul
Source of Support: None, Conflict of Interest: None
Objective: To examine the validity and reliability of the Turkish version of the Palliative Care Outcome Scale (POS). Methods: This methodological study consisted of 69 patients hospitalized in the palliative care (PC) service of three hospitals between June 2016 and August 2016, 69 carers who undertook continuously primary care of these patients, and 28 staff members working in the PC service and providing care to these patients. The data of the study were collected using the Personal Diagnosis Form, the POS. The scope, structure and criterion validity and internal consistency reliability of the scale were tested. Item analysis, Cronbach's alpha analysis, content validity ratio, confirmatory factor analysis for construct validity, criterion validity, patient validity, and intraclass correlation coefficient for the adaptation of outcomes of patient, carer, and staff groups were conducted. Results: The content validity ratio of the scale was found to be higher than 0.80 for patient, carer, and staff questionnaires. Item-total score correlation coefficients were determined between 0.27 and 0.72 for the items in the patient questionnaire, 0.33–0.67 for the carer questionnaire, and 0.34–0.72 for the staff questionnaire. The Cronbach's alpha reliability coefficients were determined as 0.64 for the patient questionnaire, 0.73 for the carer questionnaire, and 0.68 for the staff questionnaire. Conclusions: The Turkish version of the POS was determined to be a valid and reliable tool to be used for assessing the needs of PC patients in three dimensions in terms of the perspectives of patient, carer, and staff.
Keywords: Carer, factor analysis, hospice, palliative care, patient
|How to cite this URL:|
Kocatepe V, Kayıkçı EE, Saygılı &, Yıldırım D, Can G, Örnek G. The Palliative Care Outcome Scale: Turkish validity and Reliability Study. Asia Pac J Oncol Nurs [Epub ahead of print] [cited 2020 Feb 27]. Available from: http://www.apjon.org/preprintarticle.asp?id=276945
| Introduction|| |
Palliative care (PC) is a relatively new specialty that has evolved during the last five decades. Each year, an estimated 40 million people need PC. Seventy-eight percent of these people reside in low- and middle-income countries. There are many advantages of PC. Patients and their families have reported an enhanced quality of life. Moreover, distressing symptoms such as pain, depression, and spiritual distress have been better controlled.,,
As a result of the increase of PC centers, the necessity of evaluating care outcomes has arisen. Outcome measurement is a method used for measuring the changes in a patient's current and future health status that can be associated with the preceding healthcare. Outcome measurement involves the use of a measure to establish a patient's baseline health status and then assesses the changes over time against that baseline status. Outcome measures allow us to record these changes and enable us to measure the structure, process, and output of care.
Having been translated into more than 12 languages, the Palliative Care Outcome Scale (POS) is frequently used in PC units.,,,, The POS was developed in 1999 for advanced stage cancer patients., It is used not only for cancer patients but also for patients with HIV/AIDS and chronic diseases and not only with patients but also with carers and staff members., It has been used as a research tool and for informing clinical practice, monitoring service interventions, and assessing and enhancing the quality of care., A symptom module (the POS-S) was developed as an addition to the POS to describe the effect of PC-specific symptoms.
The number of PC centers is increasing in Turkey. However, these centers do not have a valid and reliable scale to evaluate patient outcomes. The POS has three versions. These three versions are all guided by clinical experiences and patients' general condition. The POS enables two types of scores to be generated. First, scores can be related to individual question items. Individual item scores can enable staff members to monitor change over time and focus on particular patients. Second, scores can be summarized. The summary score is generated by the total score possible, which can only be generated when interventions have not made any significant impact.
There is no published study using a Turkish POS as an assessment tool in Turkey. The aim of this study is to investigate the validity and reliability of the Turkish version of the POS.
| Methods|| |
In this study, the POS and the POS-S were adapted into Turkish, and their validity and reliability were methodologically assessed.
The population of this study was composed of the PC patients receiving treatment in three hospitals and the carers and staff members providing care for these patients. The study included 69 patients who were conscious and receiving treatment in a nonintubated PC unit. In addition, 69 carers who carried out the continuous primary care of the patients and 28 staff members working in PC service and providing care for these patients were included in the study. In previous validity and reliability studies, it was recommended for the sample size to be 5–10 times greater than the total number of scale items. There are 10 items in the POS, and the aim was to reach five times the number of patients, carers, and staff members.
The personal identification form, developed by the researcher, the POS, and the – Palliative care Outcome Scale- Symptom Scale for Palliative Care (POS-S) were used to collect the data.
The personal identification form was created separately by the researcher for the patients, the carers, and the staff members based on the literature. The personal identification form for staff members is an information form that includes questions about personal characteristics and knowledge about PC. The personal identification form for patients is composed of 11 questions about the personal characteristics of the patients, characteristics related to the disease process, and knowledge about PC. The personal identification form for the carer composed of 10 questions, including information about personal characteristics and knowledge about care.
The POS is used to measure outcomes by evaluating many important and necessary requirements in the PC of advanced stage patients. The POS was developed by Hearn and Higginson in 1999. Each item is scored using a 5-point Likert scale. The 10 items consist of questions such as physical symptoms of the patient, psychological needs (emotional and spiritual) and determining information and practical support. The assessment tool also has two questions, one of which is open ended, evaluating the performance status of the patient. The POS scale has 3 versions: patient, carer, and healthcare professional. A high score on the scale indicates high PC needs.
POS-S is an assessment tool composed of 11 items that evaluate the patient's severity of symptoms. Symptom severity was measured using a five-point Likert scale, with each item rated from “0” meaning “none” to “4” meaning “very severe.”,,
The data for this study were collected between June and August 2016 in the PC units of three hospitals.
In the study, the number, percentage, mean, and standard deviation were used for the descriptive characteristics of the participants and the descriptive statistics of scale scores. P < 0.05 was considered as statistically significant.
Pearson correlation analysis (reliability analysis) for item analysis (item-total score analysis), Cronbach's alpha analysis (reliability analysis) for scale internal consistency, confirmatory factor analysis (CFA) (validity analysis) for construct validity, Pearson correlation analysis (validity analysis) for criterion validity, intraclass correlation coefficient (ICC) for the adaptation of the outcomes of patient, carer, and staff groups were used. Repeated measures analysis of variance was used (advance analysis: Bonferroni test) to compare the POS mean scores of the three groups.
To conduct this study, written permission was obtained from the Research Project and Coordination Assistant on behalf of the website “A resource of Palliative Care” (https://pos-pal.org/) from which the scale was taken. Approval from the Ethics Committee was obtained to conduct this study (Approval No. 2016-4/19).
| Results|| |
Of the 69 patients, 34 (49.3%) were female, and 35 (50.7%) were male. The mean age of the patients was 69.28 ± 14.34 years. Of the patients, 34 (49.3%) were primary school graduates, and 55 (79.7%) were married. Thirty-seven (53.6%) of the patients had cancer, 15 (21.7%) had neurological diseases, 9 (13%) had respiratory diseases, and 8 (7.2%) were hospitalized due to other disease diagnoses. The mean duration of hospitalization of the patients in the PC unit was 12.38 ± 14.07 years.
Of the 69 carers participating in the study, 51 (73.9%) were female, and 18 (26.1%) were male. The mean age of the carers was 47.45 ± 13.00 years.
Fifty-nine (85.5%) of the staff members were female, and 25 (36.2%) were married. The mean age of the staff members was 27.43 ± 5.57 years. Of the staff members, 49 (71.0%) had a bachelor's degree, and 5 (7.2%) had a graduate degree. The average working duration of the staff members in the PC profession was 4.57 ± 5.22 years. A total of 64 (92.8%) of the staff members were working in PC for the last year.
Language and content validity
The scale was translated into Turkish by three linguists who had a good command of English. The final version of the scale items, which was formed by selecting the most appropriate expressions from the Turkish translations, was backtranslated into English by a linguist whose mother tongue was Turkish and who was informed about this subject in detail. The researchers made the necessary revisions by selecting the most appropriate expressions from the Turkish translation of the scale, and the finalized scale items were presented to the three experts for their opinions. The experts were given both the original version and the translated version of the scales and asked to score the items between 1 and 4 (1 = many changes are needed, 4 = very appropriate) to determine their appropriateness. The items were then revised based on the feedback from the experts. The Turkish version was then backtranslated into English by a linguist who was fluent in both Turkish and English. The content validity examines how much the scale covers the basic elements of the structure requested to be measured. The content validity ratio was developed by Lawshe (1975). According to the results, content validity ratio of the scale items was found to be between 0.80 and 1.00.
Pilot testing of the scale, language, and scope validity was conducted with five patients, 5 carers, and 5 staff members who were not included in the study. Following the pilot study, any required changes were made in line with the feedback provided by the participants, and the items took their final form.
Descriptive statistics of the Palliative Care Outcome Scale items
[Table 1] shows the lowest and highest scores obtained from the patient, carer, and staff POS items and the mean score and standard deviation of each item.
|Table 1: Descriptive statistics of the palliative care outcome scale items|
Click here to view
Reliability analysis of the Palliative Care Outcome Scale
Item analysis of the Palliative Care Outcome Scale
Item scores and total score correlations of the POS translated into Turkish and total score correlations were evaluated via Pearson correlation analysis [Table 2].
|Table 2: Item-total score correlations of the palliative care outcome scale|
Click here to view
When examining item-total score correlations of the items for the reliability of the POS (Patient Questionnaire) results, it was determined that the correlation reliability coefficients of 10 items (Pearson correlation/Pearson product-moment correlation) were between r = 0.27 and 0.72, positive, and statistically significant [P < 0.05 for items 6, 9, and 10; P < 0.001 for the others, [Table 2].
When examining item-total score correlations of the items for the reliability of the POS (Carer Questionnaire) results, it was determined that the correlation reliability coefficients of 10 items were between r = 0.33 and 0.67, positive, and statistically significant [P < 0.01 for the item 10; P < 0.001 for the others, [Table 2].
When examining item-total score correlations of the items for the reliability of the POS (Staff Questionnaire) results, it was determined that the correlation reliability coefficients of 10 items were between r = 0.34 and 0.72, positive, and statistically significant [P < 0.01 for items 5 and 9; P < 0.001 for the others, [Table 2].
Internal consistency reliability analysis of the Palliative Care Outcome Scale
In the internal consistency analysis, which is one of the reliability indicators of the Turkish version of the POS, the Cronbach's alpha reliability coefficients were α = 0.64 for the patient questionnaire, α = 0.73 for the carer questionnaire, and α = 0.68 for the staff questionnaire [Table 1]. Since omitting the items with low factor loadings (path coefficients) in the CFA did not cause any significant change, there was no need to omit any item.
Alpha values obtained when the item was omitted
The alpha reliability coefficients of the overall scale when an item was omitted from the structure of the measurement tool were α: 0.55–0.67 for the patient group, α: 0.68–0.74 for the carer group, and α: 0.61–0.71 for the staff group [Table1]. Since omitting the items with low factor loadings (path coefficients) in the CFA did not cause any significant change, there was no need to omit any item.
Comparison of the Palliative Care Outcome Scale results for patients, carers, and staff members
One-way variance analysis (posthoc analysis: Bonferroni test) was used in the dependent groups for the mean scores' comparison of the patients, the carers, and the staff members for the POS [Table 3].
|Table 3: Comparison of palliative care outcome scale mean scores of the patient, carer, and staff|
Click here to view
When the mean scores of the patients, carers, and staff members for the POS were examined, it was determined that there was a very significant difference among the groups [P < 0.001, [Table 3]. In the advanced analysis, the mean scores of the patient needs obtained from the diagnosis of staff members were significantly lower than the mean scores of the patient needs diagnosed by carers (P < 0.05), and the difference between the other pairwise groups was not significant (P > 0.05).
When the concordance among the POS mean scores from the three groups was examined with an ICC (two-way random effect, consistency model), the concordance coefficient among the scores of the patients, carers, and staff members was found to be 0.70 and very significant [P < 0.001, [Table 4].
|Table 4: Confirmatory factor analysis compatibility values of the palliative care outcome scale (s=69)|
Click here to view
According to the performance states of the patients who were not scored in the study, 9 (13.0%) of the patients stated they completed the scale by themselves, 19 (27.5%) with the help of a friend or relative, and 41 (59.4%) with the help of a member of the care team. Fourteen (20.3%) of the carers expressed the activity status of their patients as limited, 8 (11.6%) as outpatient, 25 (36.2%) as limited in care, and 22 (31.9%) as entirely dependent. The staff members stated that 5 (7.2%) of their patients were completely active, 28 (40.6%) had limited activity, 7 (10.1%) were outpatient, 20 (29.0%) were limited in care, and 9 (13.0%) were entirely dependent.
Validity analysis of the Palliative Care Outcome Scale
Construct validity of the Palliative Care Outcome Scale: Confirmatory factor analysis
To confirm the compatibility of the Turkish version of the POS, CFA was performed, and CFA fit values are presented in [Table 4]. The path coefficients (factor loadings) of all items for their own factor were found to be between 0.09 and 0.68 in the patient questionnaire, between 0.16 and 0.67 in the carer questionnaire, and between 0.15 and 0.85 in the staff questionnaire.
Simultaneous criterion validity of the Palliative Care Outcome Scale: Concurrent validity
For the criterion validity of the POS, the correlation between the scale scores and the scores of POS-S measured simultaneously was examined by Pearson correlation analysis.
A strong, positive, and statistically very significant correlation was determined between the total score from the POS and scores of the POS-S (P < 0.001). As the POS scores increased, the POS-S scores increased.
The compliance of patient, carer, and staff POS results is given in [Table 5].
|Table 5: Compliance of patient, carer, and staff palliative care outcome scale results|
Click here to view
| Discussion|| |
The POS includes questions that have been used effectively in other outcome measurement scales. The items measure physical and psychological symptoms; spiritual, practical and emotional concerns; and psychosocial needs.
Generally, individual POS item scores of zero or one require less clinical attention than items that score three or four. For example, if a patient scores a four for question one when rating their pain, the patient is reporting pain that is overwhelming to them and hinders their ability to think of anything else.
In Turkey, there is no valid and reliable scale that evaluates the results of PC in three dimensions by patients, carers, and staff members. In this study, the POS and the POS-S, which were adapted into Turkish, were found to be valid and reliable measurement tools.
Discussion of reliability results
Reliability shows how accurately a scale measures the property it wants to measure as well as the productivity and continuity of the scale. There are many methods used to assess reliability. In this study, the determination of Cronbach's alpha coefficient and correlation analysis were used.
Based on the literature, a measurement tool is not reliable if the alpha coefficient from the internal consistency reliability analysis is lower than 0.40, has a low reliability if the alpha is between 0.40 and 0.59, is quite reliable if the alpha is between 0.60 and 0.79, and is highly reliable if the alpha is between 0.80 and 1.00. In this study, the scale was found to be quite reliable according to the literature.,,
The level of item-total score correlations in item selection is an important criterion., The reliability of the item-total score of these scale items is at an adequate level. Item correlations are similar to the original version of the scale and its versions applied in other countries.,,
Since omitting the items with low factor loadings (path coefficients) in the CFA did not cause any significant change, there was no need for omitting any item.
When the concordance between the POS mean scores from the three groups was examined with an ICC (two-way random effect, consistency model), the concordance coefficient between the scores of the patients, carers, and staff members was found to be 0.70 and very significant.
According to the identification of performance states of the patients who were not scored in the study, 9 (13.0%) of the patients filled out the scale by themselves, 19 (27.5%) with the help of a friend or relative, and 41 (59.4%) with the help of a member of the team. Fourteen (20.3%) of the carers expressed the activity status of their patients as limited, 8 (11.6%) as outpatient, 25 (36.2%) as limited in care, and 22 (31.9%) as entirely dependent. The staff members stated that 5 (7.2%) of their patients were completely active, 28 (40.6%) had limited activity, 7 (10.1%) were outpatient, 20 (29.0%) were limited in care, and 9 (13.0%) were entirely dependent.
Discussion of validity results
Different methods are used in the validity analysis of the scales. Some of these methods included construct validity, simultaneous criterion validity, content validity, predictive validity, and face validity. The construct validity indicates the ability of the scale to measure the related concept or the whole conceptual structure. One of the most commonly used methods for assessing construct validity is factor analysis. Factor analysis is a process conducted to evaluate whether the items in the scale are collected under different dimensions.
Construct validity of the Palliative Care Outcome Scale: Confirmatory factor analysis
For an item to measure a structure or factor well, the factor loading should be 0.30 or above. In the CFA, the goodness of fit statistics should be at the desired level. In this study, the fact that the Chi-square results of the patient, carer, and staff questionnaires of the POS adapted into Turkish was not significant according to the result of the CFA, and the value obtained by dividing the Chi-square value into the degree of freedom was <2 which showed that the scale structure was a good model.
Simultaneous criterion validity of the Palliative Care Outcome Scale: Concurrent validity
There was a strong, positive, and statistically significant correlation between the total score from the POS and the scores from the POS-S. The PC symptom scores of the patients increased as the POS score increased. The presence of a significant correlation between the POS scores and the POS-S scores of the patients indicated that the results were compatible and that the Turkish scale was valid in measuring the desired characteristic.
The r value of the correlation coefficient should be ≥0.60, and the ICC should be at least 0.70. ICC values between 0.70 and 0.84 show moderate fit, values between 0.85 and 0.94 show high fit, and values between 0.95 and 1.00 show perfect fit. According to the average difference, there is no exact similarity between the results of the three groups in this study. While the patient and carer results were close to each other, the results of the staff members were lower. According to the ICC results, there was a moderate fit among the three group results, and the fit between the groups was 70%.
| Conclusion|| |
The Turkish POS is a valid and reliable tool that can be used with patients, carers, and staff members in three dimensions for the evaluation of patients' needs. The scale needs to be tested in larger sample groups, and further studies are required. For evaluating PC results of the scale and enhancing the quality of care, its use is recommended in all PC centers and in oncology, neurology, and cardiology services where PC patients are frequently followed.
The authors would like to express their sincere appreciation to the health professionals who participated in this study and experts for their opinions.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Higginson IJ, Bausewein C, Reilly CC, Gao W, Gysels M, Dzingina M, et al.
An integrated palliative and respiratory care service for patients with advanced disease and refractory breathlessness: A randomised controlled trial. Lancet Respir Med 2014;2:979-87.
Temel JS, Greer JA, Muzikansky A, Gallagher ER, Admane S, Jackson VA, et al.
Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med 2010;363:733-42.
Gomes B, Calanzani N, Curiale V, McCrone P, Higginson IJ. Effectiveness and cost-effectiveness of home palliative care services for adults with advanced illness and their caregivers. Cochrane Database Syst Rev 2013;6:CD007760:1-279.
Bausewein C, Fegg M, Radbruch L, Nauck F, von Mackensen S, Borasio GD, et al.
Validation and clinical application of the German version of the palliative care outcome scale. J Pain Symptom Manage 2005;30:51-62.
Costantini M, Rabitti E, Beccaro M, Fusco F, Peruselli C, La Ciura P, et al.
Validity, reliability and responsiveness to change of the Italian palliative care outcome scale: A multicenter study of advanced cancer patients. BMC Palliat Care 2016;15:23.
Koh KC, Gupta ED, Poovaneswaran S, Then SL, Teo MJ, Gan TY, et al.
Concordance in the assessment of effectiveness of palliative care between patients and palliative care nurses in Malaysia: A study with the palliative care outcome scale. Indian J Palliat Care 2017;23:46-52.
] [Full text]
Gutiérrez-Sánchez D, Leiva-Santos JP, Sánchez-Hernández R, Hernández-Marrero D, Cuesta-Vargas AI. Spanish modified version of the palliative care outcome scale-symptoms renal: Cross-cultural adaptation and validation. BMC Nephrol 2016;17:180.
Pukrittayakamee P, Sapinum L, Suwan P, Harding R. Validity, reliability, and responsiveness of the Thai palliative care outcome scale staff and patient versions among cancer patients. J Pain Symptom Manage 2018;56:414-20.
Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: The palliative care outcome scale. Palliative care core audit project advisory group. Qual Health Care 1999;8:219-27.
Bausewein C, Le Grice C, Simon S, Higginson I; PRISMA. The use of two common palliative outcome measures in clinical care and research: A systematic review of POS and STAS. Palliat Med 2011;25:304-13.
Harding R, Karus D, Easterbrook P, Raveis VH, Higginson IJ, Marconi K. Does palliative care improve outcomes for patients with HIV/AIDS? A systematic review of the evidence. Sex Transm Infect 2005;81:5-14.
Aspinal F, Hughes R, Higginson I, Chidgey J, Drescher U, Thompson M. User's Guide to the Palliative Care Outcome Scale. Vol. 10. London: Palliative Care and Policy Publications; 2002;. https://pos-pal.org/maix/resources.php
. [Last accessed on 2019 May 11].
Esin N. Data collection methods and tools and reliability and validity of data collection tools. In: Nahçıvan N, Erdoǧan S, Esin N, editors. Research, Process, Application and Critical in Nursing. İstanbul: Nobel Press; 2015. p. 193-235.
Yurdugul H. Use of Content Validity Indexes for Content Validity in Scale Development Studies. Paper Presented at: XIV. National Congress of Educational Sciences; 2005. p. 771-4.
Akturk Z, Acemoglu H. Reliability and validity in medical research. Dicle Med J 2012;39:316-9.
Akgül A. Statistical Analysis Techniques and SPSS Applications in Medical Research. 3rd
ed. Sti, Ankara: Emek Ofset Ltd.; 2005.
Capik C, Gozum S, Aksayan S. Intercultural scale adaptation stages, language and culture adaptation: updated guideline. Florence Nightingale J Nurs 2018;26:199-210.
Beck I, Olsson Möller U, Malmström M, Klarare A, Samuelsson H, Lundh Hagelin C, et al.
Translation and cultural adaptation of the integrated palliative care outcome scale including cognitive interviewing with patients and staff. BMC Palliat Care 2017;16:49.
Dzingina MD, McCrone P, Higginson IJ. Does the EQ-5D capture the concerns measured by the palliative care outcome scale? Mapping the palliative care outcome scale onto the EQ-5D using statistical methods. Palliat Med 2017;31:716-25.
Harrington D. Confirmatory Factor Analysis. New York, U.S.A: Oxford University Press; 2009.
Alpar R. Practical Statistics and Validity - Reliability. 3rd
ed. Ankara, Turkey: Detay Press; 2014.
| Authors|| |
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]