The development of the Western Australian Haemodialysis Vascular Access Complexity instrument

Abstract Background The Western Australian Haemodialysis Vascular Access Classification instrument was developed to classify the cannulation complexity of the arteriovenous fistula or arteriovenous graft as simple, challenging, or complex. Although the instrument was developed by experts in haemodialysis nursing, the instrument had not undergone formal validity or reliability testing. Objectives Evaluate the Western Australian Haemodialysis Vascular Access Classification instrument for content validity, interrater and test–retest reliability. Design Prospective cohort study. Participants Content validity was assessed by haemodialysis nursing experts (n = 8). The reliability testing occurred in one in‐centre and one satellite haemodialysis unit in Western Australia from September to November 2019. Reliability testing was performed by 38 haemodialysis nurses in 67 patients receiving haemodialysis and 247 episodes of cannulation. Measurements Interrater and test–retest reliability assessment was conducted using κ, adjusted κ, Bland–Altman plots, intraclass correlation coefficient and Pearson's correlation coefficient. Results The final version of the instrument (n = 20 items) had individual item‐level content validity indices ranging from 0.625 to 1.00 with a scale‐level content validity index of 0.89. For both interrater (n = 172 pairs) and test–retest (n = 101 pairs), most individual variables had excellent adjusted κ (n = 33 variables), some fair to good agreement (n = 6 variables) and one variable with poor agreement. The classification of simple, challenging and complex demonstrated adjusted κ of fair to good, to excellent agreement for interrater reliability with lower levels of agreement for test–retest reliability. Conclusions This instrument may be used to match a competency‐assessed nurse to perform the cannulation thereby minimising the risk of missed cannulation and trauma.


INTRODUCTION
Kidney failure is defined as the permanent loss of kidney function wherein kidney replacement therapy (KRT) is required to sustain life.
An estimated 2.6 million people receive KRT worldwide and this is projected to double to 5.4 million by 2030 (Liyanage et al., 2015). Use of KRT through peritoneal dialysis, kidney transplantation or haemodialysis (HD) is a costly but lifesaving treatment. In Australia, HD is the treatment modality of choice for 78% (n = 9557 patients) of prevalent dialysis patients (ANZDATA Registry, 2019). Patients on maintenance HD require well-functioning vascular access (VA) to achieve effective therapy. Maintaining the patency of the VA is an important patientcentred outcome established by the international Standardized Outcomes in Nephrology initiative (Viecelli et al., 2018). Repeated missed cannulation may result in serious complications such as haematoma, infection, and aneurysm formation, which can lead to need for access revision, central venous line placement, or loss of access (Al-Jaishi et al., 2017;Harwood et al., 2017;Lee et al., 2006;McCann et al., 2009;Polkinghorne et al., 2013;Schinstock et al., 2011;Vachharajani, 2014;Van Loon et al., 2009;Wilson et al., 2010). Additionally, further cannulation attempts are painful for the patient . It is therefore important to develop a HD instrument that can measure VA cannulation complexity, so that cannulation of the VA can successfully occur on the first attempt, resulting in less risk of complications for the patient. Our previous study showed that cannulating a fistula compared with a graft, absence of a stent and matured access were associated with successful cannulation (Coventry et al., 2019). The WAHVAC includes these variables in the instrument as well as other variables that impact cannulation complexity.
Instruments that assess access complexity for the related but different scenario of peripheral intravenous catheter insertion are common (Civetta et al., 2019;Hirani et al., 2019;Pagnutti et al., 2016;Van Loon et al., 2019). There is currently no "gold standard" for grading the complexity of HD VA and no published instrument to assess HD VA complexity. Proxy measures for the complexity of HD VA may include the number of cannulation attempts, the need to ask a more experienced nurse to perform the cannulation, inability to complete the prescribed dialysis session, or use of a central venous catheter (CVC) (Coventry et al., 2019). Although the WAHVAC was developed by experts in HD nursing, the instrument had not undergone formal validity or reliability testing. For this study, a panel of VA experts were invited to participate and assess content validity testing of the instrument, and reliability testing was also conducted. Therefore, the aim of this study was to evaluate the WAHVAC instrument for content validity and interrater and test-retest reliability.

Design
A cohort study design was used to assess the reliability of the instrument.

Content validity
Content validity was conducted by an international panel of experts in HD nursing (n = 8). Experts were identified by the authors through publications in the area of HD VA, the Australian New Zealand Vascular Access Nurse network, personal communication and known expertise in cannulation. An email was sent inviting the experts to participate.

Reliability
Reliability assessments were conducted using data collected from one incentre and one satellite HD unit in Perth, Western Australia from September to November 2019. The participants were HD patients (n = 67) with a VA (AVF or AVG) and HD nurses (n = 38) who were responsible for cannulating the access. To assess interrater reliability, two nurses completed the WAHVAC instrument independently for the same patient at the same time. To assess test-retest reliability, the same nurse completed a second WAHVAC instrument for the same patient at a second time point approximately 2 weeks later.

WAHVAC
The WAHVAC (see Figure 1) has 20 items that assess access history, access assessment and relevant patient clinical history. The total instrument can be scored from 0 to 209. Each of the variable scores is added together for a total score. Total scores less than or equal to 12 are classified as simple access, scores between 13 and 20 are classified as challenging, and, scores ≥ 21 are classified as complex. The value of the score is a set value, for example, if the access was surgically created less than 3 months ago, the score is 21 and the access is then classified as a complex cannulation. We recommend the complexity of the VA be assessed monthly and reassessed after a significant change, for example, after widespread infiltration or radiological or surgical intervention.

Content validity
An international panel of experts conducted content validity assessment of the WAHVAC. Descriptive characteristics of the content validity experts (demographic data; work history, education, and HD training experience) along with the content validity scores were assessed using an electronic survey via email. In addition, the expert panel were asked to comment on the (a) clarity and wording of items, (b) comprehensiveness of the items in reflecting VA complexity, (c) any items to omit, (d) areas for possible improvements or modifications, and, (e) if they agreed with the complexity score allocated to each variable on the WAHVAC. The WAHVAC was sent to the experts on two separate occasions to review the content validity of the items on the instrument.

Reliability
A research nurse collected patient and nurse demographic data at study entry.

Patient characteristics
The patient data included demographics, medications, and access history collected via a written survey.

Nurse characteristics
The nurse data included demographics, work history, education, and HD training experience, collected via a written survey.

Episodes of cannulation
The WAHVAC was completed by the HD nurse who used nurse clinical judgement to identify if the variables were present or not present on the WAHVAC. Data on episodes of cannulation included: nurse confidence (before cannulation) with successful cannulation on a scale of 0 (no confidence) to 10 (complete confidence); if swelling, bruising or haematoma were present at the VA site; if a tourniquet or ultrasound was used; type of cannulation (area, rope ladder or both), the distance between arterial and venous needles; and if an arterial needle was inserted antegrade.
Other outcome measures included if an existing CVC was used; if the allocated nurse did cannulate; if another nurse assisted with the cannulation; if dialysis was disrupted or unable to be completed due to cannulation issues; final online clearance (Kt/V); and the number of cannulation attempts. Cannulation episode success was defined as insertion of two needles (arterial and venous) without extra attempts.
The research nurse coordinated the HD nurses to enable two nurses to independently complete the WAHVAC for interrater reliability assessment. The research nurse also coordinated the same HD nurse to complete the WAHVAC for test-retest reliability assessment after a 2-week period.

Content validity
Expert reviewer characteristics were analysed using descriptive characteristics. An a priori acceptable level of interrater agreement for relevancy was set at 0.70 and higher (Mojahedi et al., 2014). The content validity index (CVI) was computed to derive the CVI for each item (I-CVI) in the scale. The I-CVI was calculated as the proportion of experts rating either a 3 or 4 (not relevant = 1, somewhat relevant = 2, quite relevant = 3, and very relevant = 4), divided by the total number of experts who rated the item. The I-CVIs between 0.7 and 1.0 inclusive were retained, the I-CVIs between 0.5 and 0.7 were further revised or clarified, and the I-CVIs <0.5 were discarded. Derivation of the overall instrument was expressed as the number of items rated three or four by at least 80% of the experts. Scale-level content validity (S-CVI), the proportion of items given a rating of three or four by all the raters, was derived from averaging across all I-CVIs. Also reported was the number of variables where there was total expert agreement and S-CVI universal agreement (S-CVI UA), and this was calculated by dividing the number of variables with total expert agreement by the number of variables in the instrument.

Reliability
To assess the interrater and test-retest reliability of each of the individual variables (present, not present) and the classifications of simple, challenging and complex on the WAHVAC we conducted observed and expected agreements, κ statistics and their 95% confidence intervals (CI), the bias index, prevalence index and the adjusted κ value. See Supporting Information 1 for formulae calculations. Multiple studies (Byrt et al., 1993;Viera & Garrett, 2005) have reported that the κ statistic is not always satisfactory for assessing agreement, and recommend that bias and prevalence be taken into account when the magnitude of one or both of their indices is close to one. Bias, prevalence indices, and the prevalence-adjusted bias-adjusted κ (referred to as adjusted κ) were calculated (Byrt et al., 1993). For interpretation purposes, a κ (and adjusted κ) scale was used (poor, <0.40; fair to good, 0.40-0.75; excellent, >0.75) (Fleiss, 1981).
The reliability of the total score of the WAHVAC (range: 0-209) was assessed using intraclass correlation coefficients (ICCs), Bland-Altman plots, and linear regression was used to investigate evidence of proportional bias. To calculate ICC, a two-way random effect model was calculated. For the regression analysis to assess proportional bias, the variables assessed were the difference in score total between the two nurses, and the mean total score for the two nurses. A scatter plot was used to investigate the relationship between the total scores of the interrater reliability as well as the test-retest reliability, and Pearson's product moment correlation coefficient was reported. Preliminary analyses were performed to ensure no violation of the assumptions of normality, linearity and homoscedasticity.

Content validity WAHVAC
Demographic data of the HD nurse experts (n = 8) are presented in Table 1. On the first occasion, the WAHVAC instrument was sent to the experts, the original 35-item instrument was reduced to a 24item instrument. On the second expert review, the instrument was further reduced to 20 items. The wording of the items was clarified and scores for individual variables were adapted on the advice of the experts. The final instrument reports I-CVI > 0.75 for 19 items with one item at 0.62. The S-CVI was 0.89 and S-CVI UA 0.40 (see T A B L E 1 Demographic characteristics of content validation experts (N = 8)

Supporting information Appendix 1 and Supporting Information
Material 2).

Patient and nurse characteristics
The demographic data of the patient and nurse characteristics are presented in Table 2.  Table 3.

Reliability assessment
Interrater reliability The interrater reliability conducted using 172 pairs of observations are presented in The BI is the difference in the proportions of "yes" between the two nurse assessors; it has a minimum of −1 and maximum of 1.
b The PI is the difference in prevalence of "yes" and "no", prevalence being calculated as means for the two nurse assessors. PI has a minimum of -1 and a maximum of 1 and is 0 when the mean prevalence of "yes" is 50%. The BI is the difference in the proportions of "yes" between the two nurse assessors; it has a minimum of −1 and a maximum of 1. 2 The PI is the difference in prevalence of "yes" and "no," prevalence being calculated as means for the two nurse assessors. PI has a minimum of −1 and a maximum of 1 and is 0 when the mean prevalence of "yes" is 50%.

HAEMODIALYSIS VASCULAR ACCESS
| 193 between nurse 1 and nurse 2 (results not reported in tables). There were 9 of 172 occasions where a result was outside of the 95% confidence intervals (see Supporting Information Material 3). No statistical significance was found using linear regression (β coefficient = −0.09; a value close to 0; p = 0.26), indicating no proportional bias between the difference in score total between the two nurses, and the mean total score for the two nurses. There was a very strong, positive correlation of the WAHVAC total score from Nurse 1 and Nurse 2 (r = 0.81; n = 172; p < 0.001) (see Supporting Information Material 5).

Test-retest reliability
The test-retest reliability conducted using 101 pairs of observations are presented in Table 5

DISCUSSION
This is the first study to develop a valid and reliable HD VA complexity instrument. Maintenance of VA remains the Achilles' heel of HD treatment. Repeated missed cannulation results in serious complications and increases the risk of permanent loss of dialysis access Schinstock et al., 2011;Vachharajani, 2014;Van Loon et al., 2009 (Daugirdas et al., 2015). Bruising with hematoma may also compress the vessel and be painful to touch for the patient (Inglese, 2017). The mean time between the first assessment (test 1) and the second assessment (retest) was 19 days. Therefore, it is highly possible for the length of the viable vessel to change over a 2-3-week period.

IMPLICATIONS FOR CLINICAL PRACTICE
There needs to be greater emphasis on ways to achieve successful VA cannulation to promote the best health outcomes for patients on HD. VA is often referred to as the patient's "lifeline" and patency of the access has been identified as a core patient-centred outcome (Viecelli et al., 2018). The WAHVAC instrument offers a simple and practical approach to reduce the frequency of complications associated with missed cannulation by matching the cannulation complexity of access to an HD nurse with the appropriate level of experience ideally by the use of a structured competency assessment framework. Reduced access complications lead to better patient outcomes and quality of life and there may be an economic benefit through the reduced need for radiological and surgical interventions, as well as, need for central line insertion.
Unsuccessful cannulation is burdensome for both the patient and the nurse.
In addition, the WAHVAC gives HD centres and nurses a framework and scope by which to develop education programs and competency assessment in dialysis access cannulation. At our centre, the HD nurses are assessed and deemed competent to cannulate a "simple", "challenging" or "complex" VA. Along with competency assessment, there is ongoing nurse education and training to ensure best practice based on the latest research data and incorporating new techniques such as the introduction of ultrasound-guided cannulation as technology advances.

Implications for research
Further research is required to investigate the reliability of the variables that have low prevalence. To enhance the generalisability, external validity and applicability, the reliability of the instrument should be assessed in other HD units.

Strengths and limitations
The strengths of this study include the rigour associated with the content validation process to identify the variables associated with complexity of cannulation. Additionally, the reliability data was collected over two sites with a large sample size of both patients and nurses which increases generalisability. We had 172 pairs of observations to assess interrater reliability and 101 pairs of observations to assess test-retest reliability. A limitation of our study is the low prevalence of some of the items on the WAHVAC instrument which limited assessment of reliability for these variables. For example, there were no patients in the buttonhole establishment phase which can be explained by it being a technique not common in Australia and predominantly used in Europe and Japan (Vachharajani et al., 2020). Another limitation to the study, is the number of area cannulations used, as area cannulations weaken the fistula wall (Parisotto et al., 2014). It is important to avoid area cannulations whenever possible. There were also no thigh, necklace or brachio-basilic fistula/graft. Additionally, the patients rarely had variables of surgically created less than 3 months, first cannulated less than 3 months, nonneedleable stent in situ, multiple collaterals, nonpalpable (deep) and no further surgical access. Therefore, we recommend further research is needed to establish reliability of the variables of WAHVAC with a low prevalence.

CONCLUSION
People with kidney failure require KRT to sustain life. Successful cannulation of vascular access is required to deliver HD treatment.
This classification instrument allows patients to be matched to a competency-assessed nurse to perform the cannulation thereby minimising the risk of missed cannulation and trauma.
This study has demonstrated the WAHVAC to be a valid and reliable instrument to assess the complexity of cannulating HD VA.
Reliability agreement for individual variables of the WAHVAC varies from fair to good, to excellent with only one variable considered as having poor agreement. Although most variables were found to be valid and reliable, further studies are needed to establish the validity and reliability of low prevalence variables following the widespread application of the WAHVAC.