Die ursprüngliche wissenschaftliche Publikation zum LQ-Recorder

Es folgt die Version des Autors, welche sich geringfügig von der publizierten Fassung unterscheiden kann. Tabellen und Grafiken sind hier nicht enthalten.

Eine vollständige Wiedergabe findet sich auf der Seite Docs+Support. Dort gibt es auch Verweise auf neueres Material, weitere finden sich im entsprechenden Abschnitt der Seite mit meinem Lebenslauf.

Practical aspects of quality of life measurement: Design and feasibility study of the Quality-of-Life-Recorder and the standardized measurement of quality of life in an out-patient clinic

Jörg Sigle, Franz Porzsolt - Cancer Centre of the University of Ulm, Oberer Eselsberg, 89081 Ulm, Germany

Abstract

As quality of life (QL) data are of increasing importance, obtaining and processing of these data should be economic as well as quick, easy and standardized. For this purpose, we developed a Quality-of-Life-Recorder which is handled by the patients like a conventional paper version of a questionnaire except that an electronic pen is used. The system computes the results stores them ready for further processing and can generate a printout. Questionnaires in any language are supported. A first acceptance test with a small number of patients led to minor improvements only. The integration into a larger study focusing on patients' and staff's acceptance as well as additional personnel and management requirements was investigated. Nurses reported that patients had even instructed their successors how to use the QL-Recorder. Even elderly or slightly handicapped patients had no difficulties. No additional personnel was required for system handling or patients' instruction. This study is not representative, as flaws in patient recruitment resulted in a registration rate of only 30% of the 1100 out-patients who had visited the clinic during the study period, although the system users' age distribution resembles the national demographic curve. In the third phase of the project, we tried to collect representative data from out-patients over a defined period of time using the EORTC QLQ-C30(+3). During 19 days, 1315 patients visited the out-patient clinic and we managed to track 1267 (worst case, eq. 96.3%). Of the 1157 eligible electronic questionnaires, 1120 had matching entries in the clinic's diary, representing 94.8% of the desired patient group. Their age distribution resembles the national demographic curve. Only 1.1% of the patients declined to participate in the study. The completeness of the questionnaire data exceeded 99.96% (No question was forgotten, But in very rare cases patients said they could not answer one). Descriptive statistics, reliability and inter-scale-correlation yielded results similar to those from the literature on the validation of the QLQ-C30. We found no evidence for substantial changes in the test's psychometric parameters induced by the use with the QL-Recorder. A cost analysis showed that using the QL-Recorder, one QL-measurement with guaranteed data quality in one patient can be done at the total expense of about DM 2.50 (U.S. $ 1.80). For several sub-groups, we compared significant differences in the QL results to expectations based on clinical experience. We also performed a cluster analysis. Our investigations suggested some relationship between different areas of internal medicine and characteristic changes in QL-measurement's results.

Conclusions: Using the QL-Recorder, quality of life can be included as a routine measure in everyday clinical practice. There is no evidence for an influence on the psychometric parameters of a questionnaire. Using the QL-Recorder, we were able to obtain representative QL data from out-patients. With its open design, simple handling, possibilities for integration and its cost efficiency, the QL-Recorder is suitable for ubiquitous use.

Introduction

Although "Quality of life" (QL) has gained importance during the last years, the review of only a few articles on that subject shows that there is no broad consensus neither about the interpretation of the term nor about the methods applied for quality of life measurement. In the present work we would like to report practical experiences made with the Quality-of-Life-Recorder we developed. This instrument makes validated quality of life tests available for use as an everyday standardized routine measure. By this work, we would like to support quality of life measurement as a common element in research and practice for investigation and documentation of the success of our measures.

The development of the Quality-of-Life-Recorder included four phases: Construction and acceptance test of the QL-Recorder, investigation of acceptance and additional personnel and management requirements, collection of representative data from a large patient sample in a standardized procedure and statistical analysis of the data obtained.

As a report on the whole project can be found elsewhere (6), this article will focus on phases III and IV of the project. Theoretical aspects of QL-measurement are discussed elsewhere (1,2,3,7,8).

Methods

Collection of representative QL-data

Special requirements in the out-patient clinic

In a university's out-patient clinic, a large number of patients are distributed to a certain number of physicians and examination facilities. Reliably including all patients into a study not only requires enough hardware equipment but also depends on adequate organisation of patient recruitment. The new, temporary examination has to be integrated smoothly into established structures and processes.

Hard- and software

In a dedicated room located next to the reception desk of the out-patient clinic, we installed six Quality-of-Life-Recorders, using standard PCs in a network under Novell Netware 3.11. Each QL-Recorder ran MS-DOS, GraTaSim V. 2.91 and QLQC-33 V. 1.2. (4,5,6)

Supporting measures

In phase II, we had found that the completeness of data in a QL-assessment depended primarily on the management of the assessment and on the involved personnel's cooperation. It cannot be guaranteed that each patient visits the desk before each consultation. To cope with this situation, we supplied posters in the whole area of the out-patient clinic asking patients to visit the QL-measurement prior to each consultation. Nurses and physicians were asked to send patients who had no printout from the QL-measurement, to the QL-measurement room. There and in the waiting areas we provided a stock of papers for patient information. In the QL-measurement room we provided an additional stock of 50 copies of the paper version of the QLQ-C30(+3) for patients who would not agree with an electronic assessment but would fill in a paper questionnaire.

Personnel

During the whole sampling period, two persons from 8:00 to 10:00 a.m. and a single person during the rest of the day asked incoming patients to fill in the questionnaire, gave short instructions and stayed nearby, to help if any difficulties should occur. Any problems were documented as well as patients' reasons for disagreeing in participation. The last attendant of a day also performed the measures for quality assurance described below.

The patient's course

Inclusion- and exclusion criteria, patient selection

QL-measurement was defined to be an "examination essential for each patient who did not meet the exclusion criteria".

Exclusion criteria were: emergency, patient was bedridden or could not visit the QL-measurement for other reasons, patient did not speak German, visual problems, patient was already hospitalized, patient visited the Automatic-Implantable-Cardioverter/Defibrillator-clinic only or patient was a child.

Inpatients would visit the out-patient-area for several special examinations, for example for ultrasonic examination of the thyroid gland. All patients visiting the AICD-check were excluded a priori because the responsible personnel stated that the additional QL-measurement would disturb the course of the scheduled examinations and could not be imposed to the patients. Patients who did not agree to participate in the study were asked to present themselves in the QL-measurement room for registration of their initials and date of birth. As the QLQ-C30(+3) defines a time-frame of one week, we only scheduled a single measurement for each patient during the short assessment period of the study.

Chronological integration of the quality-of-life measurement

Most of the patients have to wait for a period of time after they have presented themselves at the desk. We considered this period of time as optimal for the QL-measurement as it would not mean additional length of stay for the patient. When patients entered the out-patient clinic, posters asked them to visit the QL-measurement before their consultation. If they had no QL printout, they were sent there from the reception desk, from the blood taking room, from several special examinations, or, finally, by the physician after the consultation.

Patients' interaction

Every patient was asked to complete the QLQ-C30 for study purposes. Participating patients were instructed how to use the QL-Recorder and completed the electronic version of the EORTC QLQ-C30(+3). Immediately afterwards they received a printout showing patient ID, time and duration of the assessment and the results of the QLQ-C30(+3), numerically as well as in a simple graph. If a patient disagreed to take part in the study, his ID and reason for disagreement were recorded. The patient received a short note confirming the registration in the QL programme.

Quality assurance

The first level of quality assurance depended on all involved personnel: all coworkers was asked to send patients without a QL-printout to the QL-measurement. For the second level, at the end of each day registered patients were compared to those who had been expected, according to the clinic's diary. If missing patients were reported, the responsible specialty was identified. Together with the involved personnel we checked whether a patient had cancelled his appointment, or whether he had not been sent to the QL-measurement facility.

In the third level of quality-assurance all patient identifications recorded in the QLQ-files with redundant entries removed were compared to all entries of the clinic's diary after the end of the study. When this comparison was finished, a several QLQ-files had no matching entry in the clinic's diaries. Therefore, after several weeks we checked in the clinic's diary whether matching appointments had been supplemented in the meantime and repeated the comparison.

Finally, we received information about the number of patients in the out-patient clinic from the clinic's data center. For each department, the number and proportion reported by the data center was compared to the number and proportion of registered patients. Moreover, the data center computed the total number of patients who visited the out-patient clinic during the assessment period. We compared this number with the number of matched QLQ-files.

Data protection

A study involving electronic data processing is required to meet substantially higher requirements in data protection than everyday clinical tools. QL-measurement was located in a separate room which was locked in off-service-hours, or continuously supervised by at least one staff member. Our fileserver was password protected and did not have a monitor nor a keyboard for security reasons. Patients' printouts were treated like other examination reports. All information was entered directly into the computer. Access-rights were set to write-only except for the required programmes. As no connection to other networks existed, information was sealed in our network as soon as the patient ticked the "Finished"-field on the questionnaire. The enhanced access-rights for the daily quality-assurance were password protected and only available in a small time-window from one hour before to one hour after the end of service-hours.

Staff poll

A poll concerning the study about 4 weeks after its completion assessed staff's opinion about additional work load for personnel and patients. Physicians were asked to rate the importance of quality of life. The questionnaire included 6 visual-analogue-scales. Results were plotted and examined for correlation using Spearman's Rho.

Organizational experiences and cost calculation

We examined the patient flow using automatically recorded data. Results might be important for the design of new QL-facilities. We also calculated the costs that might be expected for a routine QL-measurement in a patient-population comparable to the reported one.

Data analysis

We used only those QLQ-Files for exploratory data analysis which could be matched to an entry in the clinic's diary. During the assessment period, a number of students had a medical examination because of their entrance into their final year internship ("Praktisches Jahr", PJ). They were treated like a control group.

Besides commercially available software (MS Excel and SPSS), software was programmed according to the requirements for the matching of patients, separation of patients outside of SPSS, and for all other special tasks.

Psychometric analysis

We computed some values which have already been used in the validation process of the paper version of the QLQ-C30 and compared the results with those found in the literature to search for obvious influence of the electronic QL-Recorder on the questionnaire results.

Analysis of distances and Cluster-analysis

As a part of the exploratory data analysis, we applied different methods of hierarchical and k-means-cluster-analysis to check whether or not and how well different sub-populations could be distinguished.

Cluster analysis is a method which tries to generate clusters of cases of a sample. Cases within a cluster shall be as similar as possible while the clusters shall be as different as possible. The first step of hierarchical cluster analysis is to compute all distances between all patients' results. First generation clusters are made from data-sets located most closely together. Consecutively, clusters are being grouped together until all cases are agglomerated into one cluster. A report of the whole process of clustering can be plotted as a tree diagram which enables the user to identify especially distinct clusters. A simplified method of cluster analysis is k-means-cluster analysis. Here, the number of clusters to be formed is predetermined. Second, preliminary centers of the final clusters have to be specified. They are modified during the process of the analysis.

With a sample of over a thousand patients, a hierarchical cluster analysis requires a great deal of computation time and excessive storage capacity, while the resulting plot is probably difficult to be interpreted. For this reason, we did not carry out the clustering process completely. Instead, we computed the distances between all multi-dimensional QL-vectors of all patients. They were examined in a distance histogram. This method was applied using vectors of all 18 dimensions or using vectors of a few selected dimensions which were expected to distinguish the specialties.

Asking, whether results from patients of the same specialty would be similar, we finally sorted all distances between all possible pairs of patients' QL-vectors (about 624 000) in ascending order. For the n closest and the n most distant pairs of QL-vectors, we checked whether both patients of the pair belonged to the same specialty.

Characteristics of different specialties

To identify differences in QL among out-patients, we compared the results of each specialty with a control group. This control group consisted of all included patients except for the PJ-students and the patients of this particular specialty. Observed characteristics of the subgroups were compared with expected clinical pictures. For this analysis, we compared the arithmetic means of the sub-populations and computed the two-sided significance level for the difference of means using the non-parametric Mann-Whitney-U-Test.

Results

Descriptive Statistics

Assessment period, number of patients, assessment rates

Patients were included into the Ambu2-study during 19 days. During this period of time, 1315 patients were recorded in the clinic's diary, including appointments that had been entered up to four weeks after the end of the study. Out of these, 101 patients had to be excluded and 33 were inpatients. The resulting number of patients in our target group was 1181.

During the same period of time, we obtained 1156 electronically completed non-redundant QLQs and one QLQ completed on paper. Out of these, 1120 had a matching entry in the out-patient-clinic's diary. 13 patients had not agreed to take part in the study.

In the worst case, completely ignoring the QLQ-files that could not be reliably matched with entries in the clinic's diary, the resulting registered proportion of the target group was 1133/1181=95.9% and the proportion of the target group represented by completed questionnaires was 1120/1181=94.8%.

The data of the 1120 patients with matching QLQ-file and entry in the clinic's diary served as data-base for further analysis.

Age distribution

The age distribution of the patients in the study resembles the German demographic curve except for very young children (Fig. 1). Children are admitted from the childrens' hospital to our hospital only for special diagnostic procedures. Besides, the QLQ-C30 was not designed for young children.

Compliance and completeness of the questionnaires

Out of 1133 registered patients, only 13 disagreed to take part in the study while 1120 agreed. The resulting compliance was 98.9%. Patients who did not agree to participate, usually not even visited the room of QL-measurement. The most frequent reason was being afraid of delaying one's appointment. An elderly patient was afraid of disadvantages resulting from giving "wrong" answers. One patient who did not want to participate because of a "general suspicion against questionnaires", changed his mind and completed all questions very carefully. Half an hour later he came back and wanted to correct some of his answers after he had studied the printout.

The completeness of questionnaires was greater than 99.96%. Missing data did not result from overlooked questions, but some patients said, they could not answer single questions.

Distribution of excluded patients

The frequencies of patients meeting the different exclusion criteria are shown in Table 1. Almost 70% of the exclusions occurred due to language problems or because the infrastructure of the AICD-clinic, which they visited, was not compatible with the integration of QL-measurement. As a single patient may meet more than one criterion, the total number is 114.

Evaluation of the staff poll

The staff poll showed that most staff members thought, they were well informed. They thought that QL-assessment was important, that the measurement was no major burden for the patient and that their own additional workload was rather small. We found that people who thought they were well informed usually talked to patients more (r=0.63, p=0.003) and thought that QL-measurement was more important (r=0.44, p=0.045) than those who thought they were not well informed. Interestingly, we found that the physicians' answer "I payed attention to the QL-results in the patient file" correlated with "I think QL-measurement is important" (r=0.75, p=0.048) and correlated negatively with "I got additional work load from the study" (r=-0.72, p=0.055).

Integration of the study in the out-patient clinic's environment

Patient flow varies over the day

The frequency of patient visits to the QL-measurement room versus time of the day is shown in Figure 2. It shows that there are peak values of 25 patients per hour in the morning and a decrease to only 1/5 to 1/10 of this flow in the afternoon. This phenomenon influenced the QL-Recorders' system load: while we needed 4 to 6 systems around 8:00, 9:00 and 10:00, a single system was usually sufficient in the afternoon. With an average completion time of 5.5 minutes, only very small delays occurred.

Cost and personnel requirements

A cost calculation for 5 years of quality of life measurement is shown in Table 2. Considering 90 000 QL-measurements during a period of five years, we calculated the total costs for a single assessment to be below US $2.00.

This calculation does not include costs for planning a study or for evaluation of the results.

Data analysis

Descriptive statistics, reliability and inter-scale-correlations

All EORTC-C30 (+3) dimensions yield results ranging from 0 to 100, but with different resolutions.

The results of descriptive statistics and reliability for the complete sample are shown in Table 3. The results use the complete possible range in all dimensions. The quartiles show that dimensions assessing function are left-skewed while symptoms are right-skewed. Cronbach's a greater than 0.7 in most scales and still around 0.6 in KF (0.66), PF (0.62), NV (0.57). Only the RF-scale has an a of 0.36.

For a comparison with literature (1) we computed inter-scale-correlations according to Pearson and found (all p-values<0.005): Correlations between function scales ranging from 0.27 to 0.7. The strongest correlations were found between RF and NRF as well as SF and NRF. Correlations between global measures ranged from 0.29 (with symptoms) to 0.65 (with functions) with the strongest correlations between QL and NQL. Correlations between symptom scales ranged from 0.04 to 0.52, preferring the area around 0.3. The scale FA showed the strongest correlations with symptoms (around 0.3) as well as with functions (around 0.6). Obstipation, diarrhea and financial impact only showed correlations in the range of 0.1 to 0.2.

Analysis of distances and Cluster-analysis

The distances between PJ-students were short even when all 18 dimensions were included, and in a distance histogram they appeared as a sharp peak. Excluding the PJ-students, other subgroups of patients produced broader peaks only when vectors constituent of a small number of dimensions were used. If additional dimensions were added, these peaks melted into a normal distribution. For example, a peak resulting mainly from cardiac and pulmonologic patients could be distinguished well using vectors of age, physical function and dyspnea, while it disappeared in a distance histogram of vectors of all 18 dimensions.

These results indicated that the postulated clusters would be distributed over all the space available with large overlapping. This was expected as the resolution of the dimensions of the QLQ-C30(+3) is quite low. Because of this, in 1120 cases there is a high probability that each sub-population's results cover the whole available range in each dimension.

Using several values for n in the range of 10 to 1000, we observed that in every case there were more pairs of patients from the same specialty among the n closest pairs of vectors than among the n most distant pairs of vectors.

As a hierarchical cluster-analysis requires a great deal of computation time and excessive storage-capacity, and as the results of the grouping process in such dimensions were not expected to be interpretable, we did not run a complete automatic grouping, but performed a k-means-cluster-analysis instead. This shortened method - depending on the initial values - could identify clusters, but it could not generate any clusters exactly resembling groups of patients from different specialties.

Search for characteristics of groups sorted by specialties

For some selected specialties, the following diagrams show how their results differ from all other patients.

M. Crohn / Colitis Ulcerosa

This sub population (Fig. 3) differs from other patients by lower average age, more fatigue, more pain, more appetite loss, more diarrhea. These characteristics fit the clinical picture of both diseases, i.e. an age peak between 20 and 40 years, inflammatory disease of the intestine with abdominal pain, diarrhea and fatigue.

Cardiologic patients

This sub population (Fig. 4) differs from other patients by higher average age around 60 years, worse physical function, lower global quality of life, more dyspnea. These findings reproduce the clinical picture of cardiologic patients.

Pulmonologic patients

This sub-population (Fig. 5) differs from all other patients by: moderately higher average age, reduced emotional function, reduced social function, fatigue, appetite loss, obstipation, differences in physical function, dyspnea and pain are not significant, but a larger sample might lead to a better characterization. The reduction of emotional function could be explained by anxiety caused by episodes of asthma. Appetite loss is a common symptom in pulmonal diseases. Obstipation is one of the most obvious differences between pulmonologic and cardiologic patients. It might be explained as a side-effect of the medication for gastric ulcer prophylaxis during corticoid-therapy.

Bone-marrow-transplantation-patients (BMT)

This sub-population (Fig. 6) differs from all other patients by lower average age, around 37 years, reduced role function better emotional function, cognitive function and global quality of life, more financial difficulties. The lower average age would be explained sufficiently by the lower transplantation risk of younger patients which enables to receive this treatment more frequently. Differences in the dimensions EF, CF and QL might be caused by the selection process associated with the procedure of BMT and/or by special emotional support received by these patients.

Discussion

Experiences made in the course of the present study

Advantages and disadvantages of the used system

Even during the first observations of patients' interaction with the QL-Recorder, it appeared to be such a simple Man-Machine-Interfaces that virtually no patient has any problems in its handling. While a patient is filling in a questionnaire, the PC remains completely in the background. The patient focuses on the questionnaire which he is familiar with. Only after the patient has finished the questionnaire, the computer turns up again: either indicating missing answers, or confirming the completion of the test.

The QL-Recorder has a number of disadvantages as well as advantages. Disadvantages are, for example, the size of the tablet which makes transportation difficult. It could be substituted by a laptop computer with a digitizer integrated in it's screen. While multiple page questionnaires require a turning of pages on the tablet, a screen's contents could change considerably faster. On the other hand, the large size of the tablet guarantees that the questionnaire is clearly readable even by patients with visual problems. The QL-recorder saves the patient from having to learn how to handle a new data entry tool (like a mouse) and from problems in adapting to continuously changing information on a screen. The most obvious advantage is that patients do not feel they are using a computer but that they are completing a "paper-like" questionnaire.

Compared to more expensive systems, for example pen-based laptop-computers or personal-digital-assistants with their required hard- and software-infrastructure, the low costs, the use of fully-developed techniques and standard-hardware and the ability to integrate in current electronic-data-processing-environments are other advantages.

Experiences gained from the project

The goal of the project was to integrate QL-measurement in the daily work of the whole out-patient-clinic with its different subdivisions. We wanted to assess all patients and we were using a new method. Consequently, cooperation with nursing personnel and physicians was absolutely indispensable. The widely varying patient-flow could be handled very well, and the time required for the completion of a questionnaire was about half as long as reported in literature (1) for the paper version. During four weeks, a data set was generated which is representative for our out-patients. Patient compliance was excellent. The age-distribution of the registered patients shows that even aged patients could use the QL-Recorder without difficulties.

Results of the staff poll suggest that interest and motivation of the involved personnel influence their perception of additional work load, their estimation of stress induced to patients and their estimation of the importance of QL-measurement. After all, answers supporting QL-measurement were predominant, indicating its acceptance by the involved personnel.

During the whole study, no severe problems with the electronic questionnaire itself occurred.

These findings demonstrate that the QL-recorder can be integrated into clinical practice.

Results from further data analysis

The results of descriptive statistics, of reliability analysis and of inter-scale-correlation resemble the data from the validation process of the QLQ-C30 as reported in literature, without being completely identical(1). As the observed differences were non-systematic, there was no evidence for a change in the questionnaires' characteristics induced by the electronic instrument.

The low Cronbach's a for the RF-scale reproduces the findings reported in literature. The low value of NV might be influenced by the fact that this symptom was quite rare in our population (average=9.8; 75% reported values below 17). PF and KF approach 0.7 while all other scales exceed it. The very high values of 0.89 and 0.91 for the QL and NQL scales might indicate that the people in our population identify quality of life with their physical condition or general health constitution to a high degree.

The experiences with the new scales NRF and NQL reported on the November 1994 meeting of the EORTC Study Group on Quality of Life in Trondheim could be reproduced: both scales show a better reliability than their predecessors.

In the comparison between characteristic results of sub-populations and their clinical pictures, every characteristic result corresponded with the clinical picture of its sub-population. This finding supports the reliability of the questionnaire results. In cluster analysis, the results of pair analysis of distance vectors support the hypothesis that patients from the same specialty have similar results with a higher probability than patients from different specialties. Nevertheless, the questionnaire cannot separate the specialties distinctly throughout all dimensions, as the k-means-cluster analysis showed. These observations suggest that similar clinical pictures may cause similar symptoms or functional impairment which results in better correlation among results from patients of the same specialty versus results from patients of different specialties.

Conclusion

We demonstrated that the Quality-of-Life-Recorder can be used to collect representative QL-data from patients in an out-patient clinic. No evidence was found for a significant influence of the QL-Recorder on the psychometric parameters of the EORTC QLQ-C30(+3). QL-measurement was well accepted by staff and patients; patient compliance was high (98.9%). The QL-Recorder can help to save time, to save money and improves data quality. It proved to be a useful tool for scientific use as well as for daily clinical practice.

Some distinctive differences between QL-results of several sub-populations of our out-patients were found. The characteristics of each sub-population corresponded with its predominating clinical pictures. This finding and results of cluster analysis supported the hypothesis that although it may be impossible to identify reliably the underlying disease from a single QL-measurement, its results have clinical validity and value.

We'd like to thank the following persons and institutions:

M. Fink, B. Mihanovic, G. Lutz, B. Schäffler for taking good care of the patients, Mr. Rettenberger and the staff of the out-patient clinic of the Medizinische Klinik der Universität Ulm for their extremely fine cooperation, W. Voigt and Mr. Ziesel for additional technical equipment, Dr. Kuhn who provided data from the clinical data center, I. Brand and S. Goldmann for their work as secretaries of the Cancer Centre of the University of Ulm, Fa. SPSS GmbH Munich for providing the SPSS-Software in time, A. Coates, W. Gaus, H. Heimpel and D. Machin for their advice in statistical questions, W. Streibl for early support of my computing ambitions, A. Piffel for criticism and suggestions and H. Sigle for improving proofreading,

The reported work is part of a doctoral thesis prepared by cand. med. Jörg M. Sigle at the University of Ulm, Germany under the supervision of Prof. Dr. med. F. Porzsolt.

Literature reference

1 Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, Filiberti A, Flechtner H, Fleishman SB, de Haes JCJM, Kaasa S, Klee M, Osoba D, Razavi D, Rofe PB, Schraub S, Sneeuw K, Sullivan M, Takeda F for the EORTC Study Group on Quality of Life: The European Organization for Research and Treatment of Cancer QLQ-C30: A Quality-of-Life Instrument for Use in International Clinical Trials in Oncology. Journal of the National Cancer Institute 1993; 85(5): 365-376

2 Dawes, RM. The robust beauty of improper linear models. Am Psychol 34:571-582, 1979

3 Osoba D, Aaronson NK, Till JE: A practical guide for selecting quality-of-life measures in clinical trials and practice. In Osoba D (Hrsg): Effect of cancer on quality of life. CRC Press, Boca Raton, 1991, 89-104

4 Sigle, JM: SC-JMS GraTaGen and SC-JMS GraTaSim software manuals, Jörg M. Sigle, Kunstvolle EDV & Elektronik 1990 - 1995

5 Sigle, JM: SC-JMS GraTaSim / Lebensqualität - Ein angenehmes System zur computergestützten Erfassung der Lebensqualität, QL-Recorder software manual, Dr. med. Jörg M. Sigle, Kunstvolle EDV & Elektronik 1993-1995

6 Sigle, JM: Praktische Aspekte der Lebensqualitäts-Messung: Die standardisierte Messung der Lebensqualität bei Ambulanzpatienten mit einem elektronischen Lebensqualitäts-Recorder, Dissertation zur Erlangung des Doktorgrades der Medizin der Medizinischen Fakultät der Universität Ulm, 1995

7 Till, JE: Uses (and some possible abuses) of quality-of-life measures. In Osoba D (Hrsg): Effect of cancer on quality of life. CRC Press, Boca Raton, 1991, 137-154

8 Ware JE and Sherbourne CD: The MOS 36-item Short-Form Health Survey (SF-36). I. Conceptual framework and item selection. Med Care 1992; 30:473-83