Does Length of Service Predict Adherence to the iCCM Diarrhoea Case Management Protocol?

Introduction

Diarrhoeal disease remains a major childhood health concern in Kenya's arid and semi-arid lands (ASALs), where limited access to water, sanitation and hygiene (WASH) services increases transmission risk. In Turkana County, the under-five mortality rate is 55 per 1,000 live births, compared with 41 nationally, while diarrhoea prevalence among children younger than five is 17.6%, compared with 14.3% nationally. Only 42% of Turkana households have access to basic drinking-water services and 9% have basic sanitation (Kenya National Bureau of Statistics et al., 2023).

Kenya's Community Health Policy 2020-2030 positions Community Health Promoters (CHPs) as frontline providers of integrated community case management (iCCM). For childhood diarrhoea, CHPs assess danger signs and dehydration, provide oral rehydration salts (ORS) and zinc, counsel caregivers, and refer children who require facility-based care (Government of Kenya, Ministry of Health, 2020a). The effectiveness of iCCM therefore depends on completing each indicated step correctly and consistently.

Evidence from sub-Saharan Africa nevertheless suggests a known-do gap between what community health workers know or report and what they do during care. In Cameroon, many community health workers did not correctly identify zinc as a treatment commodity for childhood diarrhoea (Chi et al., 2023). Self-reported measures are also vulnerable to recall error and social desirability bias. In Burundi, the Democratic Republic of Congo, and Nigeria, provider knowledge explained only part of the observed non-adherence to childhood-illness guidelines (Clarke-Deelder et al., 2025). Similarly, an Ethiopian study found accurate dehydration knowledge among more than 90% of health extension workers but accurate practice in only about half of observed consultations (Daka et al., 2023). Direct evidence is therefore needed to determine whether reported knowledge translates into protocol-adherent care.

Structured observation records care as delivered and can identify errors that knowledge tests or self-reports may miss, although observation itself can influence provider behaviour (Griffiths et al., 2019). Previous observation-based studies have documented both strengths and gaps in community health workers' treatment and referral practices (Miller et al., 2018).

In mobile pastoralist settings such as Turkana, dispersed households, frequent movement and limited literacy complicate service delivery, training and supervision. Yet direct evidence on CHP management of childhood diarrhoea in these settings remains scarce. Existing studies also suggest that experience may consolidate practical skills, although recently trained workers may retain some protocol steps more accurately (Miller et al., 2021; Rogers et al., 2023). This uncertainty makes length of service a relevant, testable correlate of adherence.

SMS-based reinforcement offers a relatively low-cost way to maintain iCCM knowledge where face-to-face training and supervision are infrequent. Digital decision-support tools have improved adherence to childhood-illness guidelines in Nigeria (McLaughlin et al., 2024), while evidence from Kenya links training and experience to community health worker performance (Rogers et al., 2023). However, direct observation has not established whether CHPs maintain high adherence after SMS-only reinforcement among mobile, low-literacy ASAL communities.

This study used structured observation to assess iCCM diarrhoea case management in Kakuma Ward after a six-month SMS-supported reinforcement intervention. Its objectives were to (1) describe adherence to 12 protocol components and (2) estimate the association between CHP length of service and any case-level protocol deviation. Because all observed CHPs had received the intervention and the study had neither a baseline nor a comparison group, it was not designed to estimate the causal effect of SMS reinforcement.

The null hypothesis was that the odds of any protocol deviation would not differ between cases managed by CHPs in the recorded 1–5-year service category and those managed by CHPs in the two longer-service categories (5–10 years and above 10 years) combined (H₀: OR = 1). The alternative hypothesis was that cases managed by CHPs in the recorded 1–5-year category would have higher odds of deviation (H₁: OR > 1). Associations with age, education and gender were examined as exploratory covariates.

Materials and Methods

Study Design

This article reports the cross-sectional structured-observation component of a larger mixed-methods study of an SMS-supported iCCM intervention. Observations captured iCCM practice during CHP-caregiver encounters after CHPs had received weekly SMS messages in English, Kiswahili, and Nga'Turkana for approximately six months. Direct observation reduced reliance on recall and self-report but did not eliminate the possibility that observation influenced behaviour.

The analysis was cross-sectional: observations were made at one post-intervention time point, without baseline data or a comparison group. The binary outcome was any protocol deviation, and the primary exposure was CHP length of service. Accordingly, the analysis estimates an association rather than a causal effect of either service experience or the SMS intervention.

Study Setting

The study was conducted in Turkana West Sub- County, Turkana County, northern Kenya. The sub-county is an arid and semi-arid, predominantly Nga’Turkana-speaking pastoralist setting characterised by dispersed, mobile households and a substantial reliance on community-based health service delivery. The six-month SMS-supported iCCM intervention, from which the CHPs observed in this study were drawn, had been implemented across 22 community health units in Turkana West Sub-County. Structured observations for this study’s cross-sectional component were conducted at Nadapal Community Health Unit, one of these 22 units; the sampling frame for the observed sample reported here is accordingly CHPs serving Nadapal specifically, rather than the full sub-county population of CHPs.

Population

The study population for this observational component comprised CHPs serving Nadapal Community Health Unit who had completed the six-month SMS reinforcement intervention delivered across Turkana West Sub-County’s 22 community health units. Eligible observations involved CHP-caregiver encounters for children younger than five years with diarrhoea. The final sample comprised 102 unique CHPs and 102 observed encounters. Each CHP contributed only one encounter; no CHP was observed more than once.

Sampling, Sample Size, and Eligibility

The sampling frame consisted of eligible CHPs serving Nadapal Community Health Unit and participating in the SMS-supported iCCM intervention delivered across Turkana West Sub-County’s 22 community health units. Eligible CHPs had served in a functional community health unit for at least two years, had completed the six-month SMS reinforcement period, and consented to direct observation. Eligible caregivers provided consent for their child’s encounter to be observed. During the two-month data-collection period, consecutive eligible CHP-caregiver encounters presenting for observation at Nadapal Community Health Unit were included. The achieved sample size was accordingly determined by the number of eligible encounters presenting during this fixed window rather than by a pre-specified target. A total of 102 encounters involving 102 unique CHPs were observed, with each CHP contributing exactly one encounter. The resulting sample was therefore a non-probability consecutive sample, and the CHP and encounter sample sizes were identical (n = 102). We did not perform an a priori power or sample-size calculation for the length-of-service comparison; the achieved sample reflects a pragmatic consecutive sample collected within the available two-month window rather than a pre-specified, adequately powered design. Given the very small number of observed deviation events, a post hoc power calculation would not provide reliable information and was therefore not undertaken; the length-of-service comparison should accordingly be read as hypothesis-generating rather than as an adequately powered test.

Data Collection Instruments

Data were collected with a structured observation checklist adapted from Kenya's national iCCM/Integrated Management of Newborn and Childhood Illness case-management job aid (Ministry of Health, 2022) and informed by observation-based quality-of-care methods (Griffiths et al., 2019). For each encounter, observers recorded 12 protocol components: confirmation of the child's age; assessment of diarrhoea duration, danger signs and dehydration; diagnostic classification and consistency with observed signs; correct ORS and age-appropriate zinc; complete caregiver counselling; explanation of referral danger signs; correct referral; and completion of a referral note when indicated.

Data Collection Process

Trained research assistants and Community Health Assistants at the linked facility conducted the observations. They were familiar with the local context and fluent in Nga'Turkana and Kiswahili. Observers recorded each checklist item without prompting or correcting CHPs. At the end of each observation day, completed checklists were checked for completeness, entered a structured digital dataset and exported for analysis.

Outcome Definition and Variables

Length of service was categorized as 1–5 years, 5–10 years, and 10+ years. These original category labels are retained as recorded. Because the first two response categories overlap at exactly five years, they should not be interpreted as mutually exclusive intervals reconstructed from exact service duration. For regression analysis, the latter two recorded categories (5–10 years and above 10 years) were combined and compared with the recorded 1–5-year category because protocol deviations were sparse in the longer-service categories. Exact continuous years of service were not collected in the source dataset, so a fully non-overlapping numeric threshold (for example, ≤5 vs. >5 years) could not be reconstructed retrospectively; the recorded 1–5-year versus combined 5–10/10+-year comparison used in the primary analysis is offered as the closest available non-overlapping operationalisation of shorter- versus longer-serving CHPs. A small number of CHPs recorded at exactly five years of service could, in principle, have been placed in either the “1–5” or the “5–10” response category by the original data-collection instrument.

Statistical Analysis

Descriptive statistics summarised CHP characteristics and item-level adherence. Because only five of 102 encounters contained any protocol deviation and one service stratum contained no events, conventional maximum-likelihood logistic regression was susceptible to separation and small-sample bias. Firth's penalised logistic regression was therefore used to estimate the unadjusted odds ratio for protocol deviation comparing CHPs recorded in the 1–5-year service category with CHPs in the 5–10-year and above-10-year categories combined (Heinze & Schemper, 2002; Puhr et al., 2017). Results are reported as odds ratios (ORs), 95% confidence intervals (CIs) and two-sided p-values. The primary model was specified as follows:

$$\ln \left(\frac{p_i}{1-p_i}\right)={\beta }_0+{\beta }_1{\mathrm{Service}}_i$$

A secondary exploratory Firth penalised logistic regression model was fitted to examine whether the direction of the association between CHP service category and protocol deviation changed after simultaneous adjustment for age, education, and gender. Age and education were entered as categorical indicator variables, and gender as a binary indicator. All covariates were entered simultaneously; no stepwise selection or change-in-estimate criterion was used in this model.

Because only five protocol-deviation events were observed, the number of events was substantially smaller than would ordinarily be considered adequate for a multivariable model with this number of parameters. Firth's penalisation reduced small-sample and separation bias, but it cannot compensate for the limited information available (Peduzzi et al., 1996). The adjusted model was therefore treated strictly as an exploratory sensitivity analysis. Its coefficients, confidence intervals, and p-values were not interpreted as stable estimates of independent causal effects.

$$\ln \left(\frac{p_i}{1-p_i}\right)={\beta }_0+{\beta }_1{\mathrm{Service}}_i+{\beta }_2{\mathrm{Age}}_{2i}+{\beta }_3{\mathrm{Age}}_{3i}+{\beta }_4{\mathrm{Age}}_{4i}+{\beta }_5{\mathrm{Edu}}_{2i}+{\beta }_6{\mathrm{Edu}}_{3i}+{\beta }_7{\mathrm{Edu}}_{4i}+{\beta }_8{\mathrm{Edu}}_{5i}+{\beta }_9{\mathrm{Gender}}_i$$

Ethical Considerations

Ethical approval was granted by the Kenyatta University Ethics Review Committee, with research authorisation from the National Commission for Science, Technology and Innovation and relevant national and Turkana County health authorities. Written informed consent was obtained from all participating CHPs, and separately from each caregiver specifically for their child’s care to be directly observed, before any observation began. Participation was voluntary, and confidentiality was maintained during data collection, storage and reporting.

Findings

Descriptive Findings

Note. Service-duration categories are reproduced as recorded in the source dataset. The original response categories “1–5 years” and “5–10 years” overlap at five years; exact continuous years of service were not available for retrospective recategorisation.

Among the 102 observed CHPs, 52 (51.0%) had 5-10 years of service and 40 (39.2%) had more than 10 years. The largest age group was 35-44 years (44.1%), followed by 25-34 years (29.4%). Only one CHP (1.0%) had post-secondary or tertiary education, and 52 (51.0%) were married in a monogamous civil union (See table 3).

Table 4 shows a strong service-related pattern in item-level adherence. CHPs with 5-10 years and more than 10 years of service averaged 100.0% and 99.8% adherence across the 12 components, respectively.

Note. Length-of-service categories are reproduced as recorded in the dataset; see Table 3.

The 1-5-year group averaged 85.0% adherence, with the largest gaps in correct ORS provision and completion of referral notes. Mean item-level adherence, weighted across service groups, was 98.4%; this weighting pooled each item’s adherence percentage across the three length-of-service groups in proportion to their respective group sizes (n = 10, 52, and 40) to obtain an overall adherence rate for each of the 12 items, and these 12 overall rates were then averaged. This difference should be interpreted cautiously because the 1-5-year group comprised only 10 observations.

Figure 1

Note. Length-of-service categories are reproduced as recorded in the dataset; see Table 3.

Empirical Findings

The primary unadjusted Firth model estimated a strong association between the recorded 1–5-year service category and protocol deviation, although the confidence interval was wide because only five deviations were observed (See Table 5). We fitted a secondary simultaneous multivariable Firth model as an exploratory sensitivity analysis to assess whether the direction of the service-category coefficient changed after including age, education, and gender.

Note. Outcome = any protocol deviation. Reference group = CHPs recorded in the 5–10-year and above-10-year service categories combined. OR = odds ratio; CI = confidence interval.

As a fully non-parametric complement to the Firth model, and to present the primary comparison in the exact-count, descriptive form recommended for sparse binary outcome data, a two-sided Fisher’s exact test was applied directly to the underlying 2 × 2 table of protocol deviation by service category (recorded 1–5-year group: 4 deviations among 10 encounters; combined 5–10-year/above-10-year group: 1 deviation among 92 encounters). This assumption-free test likewise indicated a statistically significant association (p = .0002), corroborating the direction and significance of the unadjusted Firth estimate reported in Table 5 without relying on any large-sample or model-based approximation.

We then fitted a follow-up Firth model, as shown in Table 6, including covariates for age, education, and gender to assess whether adjustment changed the effect of service length on protocol adherence. Because the outcome comprised only five deviation events, the model was necessarily sparse and was interpreted as a sensitivity analysis rather than as a stable multivariable prediction model.

Note. Outcome = any protocol deviation. Service-category reference group = CHPs recorded in the 5–10-year and above-10-year categories combined. Age reference = 18–24 years; education reference = incomplete primary; gender reference = male. OR = odds ratio; CI = confidence interval. Estimates are exploratory because only five protocol-deviation events were observed. The model was fitted as part of a simultaneous sensitivity analysis and should not be interpreted as providing stable, independent effect estimates. NE = not reliably estimable because of sparse outcome data/separation.

After adding the covariates to the follow-up exploratory adjusted model, the service-category coefficient remained positive (OR = 75.51, 95% CI [4.70, 1213.98], p = .002), but the extremely wide confidence interval indicates substantial uncertainty about the magnitude of the relationship. No age or education coefficient reached conventional statistical significance; however, with only five protocol-deviation events, these estimates were highly imprecise, and the absence of statistical significance should not be interpreted as evidence of no association. The gender reference category was male. The female coefficient was non-estimable because the model did not yield a stable finite estimate under the sparse outcome distribution.

Discussion

Observed adherence to the iCCM diarrhoea protocol was high, with deviations concentrated among CHPs with shorter service. Ninety-seven of 102 encounters (95.1%) were fully adherent, and weighted mean item-level adherence was 98.4%. Four of the five case-level deviations occurred in the 1-5-year service group. The unadjusted Firth estimate supported the directional hypothesis that shorter service would be associated with higher odds of deviation, although its wide confidence interval requires cautious interpretation.

One plausible explanation is that repeated exposure to childhood-illness cases helps CHPs consolidate the sequence of assessment, classification, treatment, counselling, and referral. This mechanism remains inferential, however: the cross-sectional design cannot separate the effect of accumulated experience from unmeasured differences between shorter- and longer-serving CHPs.

The direction of the service-category association was unchanged in the exploratory simultaneous Firth model, but the adjusted magnitude cannot be estimated reliably from these data. With only five deviation events relative to the number of model parameters, the adjusted coefficients were highly unstable, and some covariate effects were effectively non-estimable. The adjusted model, therefore, provides only a sensitivity check on the direction of the primary service-category association and should not be interpreted as establishing an independent effect of length of service.

The direction of the service-length association is consistent with Rogers et al. (2023), who found that training and experience predicted community health worker knowledge and performance more strongly than literacy or formal education in a Kenyan setting. The present data did not show clear associations with age or education, but those null results should not be overinterpreted because only five observations were available for analysis, and the adjusted coefficients were highly imprecise.

The high observed adherence differs from reports of substantial knowledge or practice gaps in Cameroon, Ethiopia, and other sub-Saharan African settings (Chi et al., 2023; Clarke-Deelder et al., 2025; Daka et al., 2023). Differences in measurement, case mix, supervision, workload, and implementation context may contribute to this contrast. Direct observation measures care delivery more closely than self-report, but it can also temporarily increase adherence when providers know they are being observed. Cross-study comparisons, therefore, do not establish that CHPs in Turkana are more adherent than community health workers elsewhere.

All participating CHPs had received six months of SMS reinforcement, so the observed adherence is compatible with, but does not demonstrate, a beneficial SMS effect. With no pre-intervention measure or unexposed comparison group, the study cannot determine whether adherence would have differed without the messages. Evidence from digital decision-support and refresher-training studies supports further evaluation (McLaughlin et al., 2024; Miller et al., 2021), but a controlled longitudinal design is needed to isolate any intervention effect.

Strengths and Limitations

A principal strength was the use of structured observation rather than self-reported adherence. This strength is nevertheless qualified by several limitations. First, the consecutive sampling of encounters at a single community health unit may underrepresent cases that were not recognised or referred for care, and it limits generalisation of the findings beyond CHPs serving Nadapal Community Health Unit. Second, providers may have altered their practice because they knew they were being observed, and observers were not reported to be blinded to CHP length of service, which could lead to differential recording of protocol deviations by service category. Third, the manuscript does not report a formal inter-rater reliability statistic, nor the duration or content of observer training and calibration, even though more than one observer was involved in data collection; these should be documented, or explicitly noted as unavailable, in any future revision. Fourth, the full 12-item observation checklist and item-level operational definitions (for example, of “correct” or “complete”) are not reproduced in the manuscript; making the checklist available as a supplementary appendix would allow readers to assess measurement validity directly. Fifth, the analysis included only five deviations and 10 observations in the 1-5-year group, resulting in wide confidence intervals and unstable multivariable estimates; a two-sided Fisher’s exact test on the underlying counts corroborated the direction and significance of the unadjusted result (see Findings), but the sparse-event limitation remains the single greatest constraint on inference in this study. Finally, the lack of baseline data and an unexposed comparison group prevents estimating the SMS intervention’s effect.

Conclusion

Among 102 observed post-intervention encounters, adherence to the iCCM diarrhoea protocol was high, but protocol deviations were disproportionately concentrated among CHPs recorded in the 1–5-year service category. The primary unadjusted Firth analysis indicated higher odds of deviation in this group compared with the two longer-service categories combined, although the wide confidence interval indicates substantial uncertainty regarding the magnitude of the association. The exploratory simultaneous adjusted model produced a service-category estimate in the same direction but was highly imprecise and should not be interpreted as establishing an independent effect. The sparse outcome data were insufficient to draw reliable inferences about age or education, and the gender effect was not estimable with reliable precision. Length of service should therefore be regarded as a potential marker of adherence risk rather than a proven causal determinant. The findings also do not establish an effect of the SMS intervention because baseline and comparison data were not used in this analysis.

These findings support prospective evaluation of whether newer CHPs benefit from enhanced support, including structured mentorship, supervised practice, and targeted refresher training. Future studies should use larger samples, repeated observations, and appropriate comparison groups to determine whether such strategies reduce protocol deviations and whether improvements are sustained over time.