Clinical Experience vs. Evidence-Based Practice: Do We Have to Choose?

Developing a manual for integrative practice, research and teaching

By Claudia Citkovitz

I have spent the last 13 years developing and leading an inpatient acupuncture program at NYU Lutheran Medical Center in Brooklyn, New York. At present, 12 acupuncturists are credentialed there to treat inpatients throughout the hospital. Four of us are preceptors and the rest are volunteers who treat for 4-8 hours a week over the course of a year, progressing from direct and indirect supervision to independent practice with oversight via clinical rounds and discussions. Also, about 60 students per year from master’s, DAOM and NCCAOM PDA programs come for observation for days- or week-long clinical intensives in the Neurology and Rehabilitation and Labor and Delivery units. Our studies of acupuncture during labor and delivery (2005-2006)[1] and acute stroke rehabilitation (2012-2014)[2] were each the first that I know of in the United States.

In conducting those studies, I had to negotiate the interface between research, which tends to standardize treatments for statistical analysis, and clinical practice, which orients to the particular needs of a single human being. For the second study in particular, my PhD project on stroke, I did quite a bit of research on the philosophy and construction of treatment manuals in disciplines other than acupuncture. I wanted to develop clinical guidelines that would support the complex work of patient assessment and treatment planning without ‘squishing the butterfly’ of individual responsiveness. Working with a wonderful team of acupuncturists, I developed both the treatment manual for the stroke study and a process for developing such manuals in the future, termed ‘evidence-informed manualization’. In this article, I will discuss the pros and cons of manualization itself, how it worked for the stroke study (with a fair number of stroke care specifics), and where I see the process being useful going forward in research, education, and clinical practice.

Practical Experience vs. Statistical Evidence

Personally, I started on the ‘everybody’s an individual’ side of the argument. My training before and during acupuncture school was in Tom Bisio’s first clinic in New York’s Tribeca (back when that was kind of a shabby neighborhood!). Everyone received plenty of personal care there, usually both a front and back treatment, as well as tui na and/or craniosacral therapy, bleeding cupping, moxibustion, and so on, selected on the basis of Tom’s clinical experience. During the course of my acupuncture education (first at PCOM-NY, then in China, then finishing up at Tri-State College) I remember expressing extreme and rather arrogant skepticism regarding the value of systematic treatment approaches such as Mark Seem’s. I owe him a public apology; time has shown his foresight to be extraordinary in a number of ways I never could have understood while still a student. When I opened my own practice, I did long intakes and extensive bodywork; I hand-filled herb prescriptions telling each patient what herb did what, and taught lots of qigong.

By contrast, in hospital care, where I now do all of my work, all clinical activities are supposed to be supported by evidence-based practice guidelines (at least in principle, though actual compliance is spotty[11]). These kinds of guidelines vary greatly in how they look, ranging from one-page flowcharts to book-length multi-author publications. What they have in common is that they are extensively researched documents constructed through a process of expert consensus that is usually quite lengthy and expensive. This is as it should be: directing care providers to operate consistently in a certain way (which may differ from their training or current opinion) is literally a life and death responsibility. One example is the use of tissue plasminogen activator therapy (tPA), the powerful ‘clot busting’ agent that can unplug blocked arteries in ischemic stroke. The drug does carry a risk of hemorrhage, but if administered within the first few hours post-stroke, the enormous reduction in symptoms warrants the risk for most patients. However, the effects of treatment lessen over time as brain cells die off, while risk of hemorrhage increases. For this reason, a cutoff point was initially set at 3 hours, then extended to 4.5 hours as greater volumes of data on treated patients became available for analysis. The size of the data set required to appropriately inform discussions of risk and benefit in this high-stakes decision was larger than even the largest controlled trial could provide, and far larger than the experience base of any single provider[3].

The advantage of large-scale data analysis over individual experience seems clear in the case of tPA for stroke. However, many argue that reliance on statistics has eclipsed valuable practitioner experience in areas requiring more complex or nuanced decisions. One such area is Labor and Delivery, where US birth outcomes are among the worst in the developed world[4]. Certainly, when it comes to acupuncture, most practitioners I know are opposed to using guidelines for acupuncture therapy, or at least have grave concerns. East Asian Medicine (EAM) has a huge diversity of assessment and treatment approaches, providing enormous flexibility with which to respond to what’s happening with a given patient, often at structural, energetic, and spiritual levels simultaneously. Many of our patients seek us out for exactly that flexibility and responsiveness, which provides a much-needed countercurrent to the ‘safety in numbers’ approach of most Western care. A terrific 2007 book called Thinking with Cases[5] puts Chinese medical writing in the historical context of Chinese legal reasoning and Buddhist philosophy, demonstrating convincingly that the fundamental unit of knowledge construction and transmission in all three disciplines is the case study, which simultaneously brings important general principles to life and models practitioner decision-making behavior. Reducing EAM to a set of research-approved point protocols would profoundly diminish not only the clinical care we provide, but also our teaching methods. My overall purpose during my PhD work was to create a document that outwardly fulfilled the form and function of evidence-based practice guidelines to an adequate degree, while inwardly providing practitioners with a matrix to support case-based knowledge building and transmission.

Developing a Manual for Acupuncture during Acute Stroke Rehabilitation

The goal of my PhD project was to develop and field-test a manual for acute post-stroke care. The manual needed to do three things:

1) capture the work we had already been doing, which appeared to be meeting patients’ needs[6];

2) provide the most up-to-date evidence for what works in acupuncture for stroke recovery, including clinical texts; and

3) leave room to grow as both experience and evidence develop and inform each other.

Between 2011 and 2013 I worked with five acupuncturists, meeting with each weekly for about 6 months, as we gathered information and started to put the manual together (Phase 1 of the project). We then met every 2-4 weeks for another year as we further developed the manual (Phase 2) and ran a feasibility study (Phase 3). The study followed 24 patients that we treated using the manual, as compared to 23 patients who declined acupuncture. Background, results, and a full discussion of study findings are available online in the thesis document, along with several versions of the manual: http://westminsterresearch.wmin.ac.uk/16057/1/Citkovitz_Claudia_thesis.pdf

Phase 1: Information Gathering

Our first method for information gathering was a literature review. Conventionally, the first step in making an evidence-based practice guideline would be to collect all possible relevant studies and grade them; for example, ‘A’ for the top-of-the-food-chain studies that compare other studies, B to large, well conducted randomized trials, and C–F for smaller studies with less bombproof methods. Given that the research literature on acupuncture for stroke is not remotely sufficient to guide clinical practice, I deemed that for this project it was critical to include clinical textbooks (in my system they scored ‘E’ as expert opinion, along with my class and discussion notes from stroke experts such as (Dr.) Zhu Ming Qing and Shi Xuemin, seen in the movie 9,000 Needles).

I also pulled in studies of stroke pathophysiology and acupuncture mechanisms, not normally included in practice guidelines, because an important part of understanding how to treat stroke patients is knowing what is happening in their brain at a given point in time. For example, during the first 7-10 days after most strokes, a meaningful number of brain cells are still struggling for survival, challenged by partial oxygen deprivation as well as inflammation and toxic metabolic waste from other dead or dying cells. Therefore, the absolute top priority during that time is increasing circulation and clearance of metabolic (aka raising clear yang and descending turbidity). Du20 is of course a key point for increasing cranial circulation, but clinical sources such as Shi Xuemin provide an experiential basis for including SP6/PC6, while LI4 and 11 are deeply entrenched in other clinical texts.

The hardest part of conducting the literature review was figuring out how to present the results in a clinically useful fashion. We started with a giant Excel document that spread across two notebook pages, which was comical with three of us crowded around it outside a patient room, squinting to find the recommended treatments. In the end, I cut up the spreadsheet and presented the literature review as an index by syndrome or problem (e.g. aphasia, constipation, upper extremity mobility), with each study or source summarized and presented in grade order. Later versions of the manual use the index as a resource to be consulted for difficult cases, while easier-to-use worksheets govern treatment planning for the majority of patients.

Besides reviewing the clinical and research literature on acupuncture for stroke, other forms of information gathering included self-observation during practice by the acupuncture team and discussions with our colleagues in the Rehabilitation unit, including doctors, nurses, and physical, occupational, and speech therapists. We asked the other providers how they saw the acupuncture impacting the patient and their work, positively or negatively. Working with the physical and occupational therapists, we learned a great deal about how motor and cognitive recovery unfolds over time. We also had a focus group with the nurses concerning where they saw stroke patients experiencing the most distress; insomnia and bowel function were important areas of study that emerged from this conversation, and turned out to be some of the most promising study results.

Phase 2: Drafting and Revision of Manual Forms

The second phase of manual development was the creation and refinement of the actual worksheets used to guide and record the clinical assessment and treatment planning process. This phase largely overlapped with the information-gathering phase described above. A British research resource, the Medical Research Council’s Guidance for Researching Complex Interventions[7], (MRC Guidance for short) is increasingly recognized as a useful model for acupuncture research, and it was very helpful in organizing our manualization process. The MRC Guidance suggests that researchers wanting to accurately evaluate interventions too complicated to be tightly standardized (such as sex education) need to develop the intervention and evaluation processes together, over time, in cycles of planning, implementation, evaluation, and revision. My stroke study as a whole constituted one full cycle. Within the second phase were three mini-cycles of developing and revising forms. As I discussed above, the very first version of the manual was a giant spreadsheet that nobody liked, although it held a lot of useful information. Version 2 aimed to make things much easier–it was a big, simple diagram of a person that we could mark up, (see illustration) along with a numbered outline presenting the main differential diagnoses and treatment approaches we found most useful. Importantly, this information was set out in a specific order, which evolved from our experience as we integrated our research reading and collegial discussions. The order we used is as follows:

1. Is the spirit present, or is it clouded or scattered? (We jokingly called this step, ‘first, find the patient’). In the time since this version, we have top prioritized any pain over 6/10, which itself scatters the spirit and slows recovery.
2. Is the central qi mechanism functioning? Most stroke patients present with some combination of urinary retention and constipation, or incontinence. In EAM terms, the qi mechanism is stuck closed or open, and has to be restarted before the patient can really heal.
3. Are there other constitutional factors or pathogens that need to be addressed? EAM history is rich in discussions of internal and external factors, while current research and imaging show us a wide variety of congenital and behavioral risk factors as well as with the location and nature of the stroke lesion. The rubric for constitutional assessment draws together as many of these considerations as possible, along with the approaches we’ve found most useful for addressing them. Most common, of course, are blood stasis and phlegm, with various types of underlying deficiency, excess and deficient bad behavior of liver qi, and spleen qi not managing the blood.
4. Speech and swallowing. In some cases patients want to work on walking first, but mostly difficulty with speaking, understanding, drinking, and eating are so very problematic in themselves that the patient’s healing is best served by focusing upon these issues first.
5. Balance and trunk stability. Many practitioners have learned the motor line in school and are eager to use it, but a patient with good leg strength and bad balance may actually need more assistance than others, to prevent falls. Therefore, we tend to use extra meridians and brain-microsystem ear points[8] to develop stability before promoting motor function.
6. Lower extremity mobility and sensation. We usually work on the leg before the arm, both because it often comes back more quickly, and also because getting up out of the wheelchair is a priority.
7. Upper extremity mobility and sensation. There is often a need to treat pain from subluxation of the shoulder joint due to paresis of rotator cuff muscles. In these cases, we try to balance the joint, as well as support motor recovery.

Over the course of the study we settled on 2.5 as the approximate number of issues for treatment that could be addressed in one session. This seemed to be the average of treatments with two relatively distinct issues (e.g. swallowing and leg mobility) and those with three overlapping issues (e.g. constipation, phlegm, and upper extremity mobility, all of which involve the yangming channels). Within the manual, for a given patient, the overall plan of care would be:

• A careful first intake that starts with reviewing the electronic medical record, to capture as much as possible of the patient’s current challenges and past medical history.
• Discussion with the patient (if possible) as well as palpation and observation to determine what they would like to work on, as well as what constitutional and pathogenic factors we see.
• Treatment per session that starts with updating the above chart review and conversation. The treatment plan then addresses 2-3 of the treatment priorities identified on the patient’s chart, proceeding in the order described above, unless there is some specific reason to depart.

Using this process, the team felt that, by the third version of the manual, we had found a way to provide patient care that was consistent between patients as well as from session to session for each patient, even with multiple providers. Version 3 was used in the final study, and consisted of:

• An intake sheet (which incorporates the treatment planning priorities)
• Three worksheets, one each for constitutional diagnosis and treatment, stroke recovery treatment (priorities 1-3), and stroke rehabilitation treatment (priorities 4-7).
• The Index of stroke syndromes and approaches, for use with patients who for some reason do not fit the worksheets in the first place, or ‘refractory’ patients for whom the worksheet treatments do not seem to be helping.

Phase 3: Concurrent Cohort Study

In order to evaluate the manual, we compared outcomes for patients who had been treated with the manual to those who had refused acupuncture treatment. This is not the kind of study you can use to ‘prove that acupuncture works’, since the groups were selected in a way that may strongly bias the results. However, small studies with this kind of non-randomized design are commonly used to assess ‘feasibility’ of methods for future study–can enough patients be recruited, do too many patients drop out because they hate the treatment, is there too much missing data, do the outcomes trend in the anticipated direction and so on. Our feasibility study looked at these questions and a few others, falling into three basic categories:

1) feasibility of study methods, as described above;

2) consistency and acceptability of manualized treatments; and

3) patient, family and rehabilitation staff perceptions of acupuncture effect.

We felt it was important to capture this third, qualitative piece, because qualitative research provides the opportunity to find out how patients and staff experience the treatment, which may point to the need to study completely different outcomes than originally anticipated. In fact, three of the outcome measures we used in the final study were added during phase 1 of manual development, when we learned from patients and nurses that they felt that the acupuncture helped with sleep, swallowing, and constipation. Overall, these, plus the new findings of the qualitative research, were the main successful study results.

The results of phase 3, in brief, were as follows:

• We enrolled 48 patients (25 wanting acupuncture, 23 declining it) in a year, which is enough for future study.
• There was a fair amount of missing data, especially from the 6-month follow-up assessments. Luckily, a 90-day follow-up has now been implemented by the main agency certifying hospitals, so follow-up in general should get easier.
• Statistical results trended positively for constipation, sleep, and global disability on 6-month follow-up, and these are the areas I am looking at for future study.
• As anticipated based on another researcher’s previous work, results from the Functional Independence Measure Outcomes did not differ much between groups, except in one specific subgroup where the acupuncture patients did do better (I am currently working on another article about this finding; it suggests that acupuncture may have more effect on motor outcomes in moderately severe strokes than in mild or very severe ones).
• No patients at all withdrew from acupuncture treatment, indicating that our flexible approach was very acceptable to them.
• Acupuncturists were quite consistent in their use of the manual, departing only 35 times total in 235 treatments (11 of these departures were for non-manual-related reasons like IV tape covering PC6, while another 12 were ‘refractory cases’–patients who didn’t improve with the initial treatments per the manual, so it was time to try something else)
• Qualitative results showed a wide variety of positive perceptions of acupuncture perceived, and no negative perceptions. Exciting new findings included:
  • Nurses and physical and occupational therapists reported that patients were more alert and also participated better in therapy. This is an important new finding that we will follow up on.
  • Pain was a more important consideration than we thought for stroke patients; we have now added it as a high treatment priority in the manual.
  • A lot of staff members reported improved ‘mood’, but what does ‘mood’ mean? Depression is known to complicate many if not all stroke cases, so that might be one area to explore; anxiety is also a likely candidate. At the moment, we have a very small study looking at outcome measures for both to see if they work well for our population.

Conclusions and Implications for Future Work

Pulling together this stroke manual was a great deal of work, but future manuals should be (and have been) quite a bit easier to develop. Stroke is a very complex and difficult-to-treat clinical area, and my team and I were working out the methods as we went. Since developing the stroke manual, I have taught it to over a hundred acupuncture students and practitioners in 1- and 2-day formats, which I don’t believe would have been nearly as useful without the manual’s built-in structure. In stroke, as in labor and delivery, our basic skills of diagnosis and treatment are still the main active ingredients of a successful treatment. However, those basic skills benefit greatly from some specific guidance about what to look for first, second, and third in our treatment planning. I have used this approach in teaching birth for the last 10 years. I have also begun teaching a similar stepwise approach to acute and chronic pain, and have been asked to consult on an NIH-funded study of acupuncture for chronic pain in underserved populations[9].

When working in a Western setting it is extremely beneficial to use a protocol that has some inherent consistency. To say to doctors, ‘here’s a study, I use similar procedures’ is to say, “I’m a team player, not a lone wolf”; here are the patients you can comfortably refer, here is a piece of research you can give them that describes more or less the intervention they will receive, and here are the results you can reasonably anticipate. There are genius outpatient practitioners whom I adore, refer to, and go to myself when I need it. There will always be a place for the completely individual nature of outpatient care. That said, even in the hospital, every single treatment I give is an entirely unique personal encounter. The conversation we have as I ask about their comfort and recovery progress is entirely unique and personal, even if similar questions are asked. Likewise, as I obtain and work with the qi on each needle, I am entirely in that moment with that patient. If the patient presentation is unusual or they haven’t improved with worksheet treatments, I do not hesitate to consult the index; if I feel I really can’t provide what the patient most needs on that day within what’s in the index, then I have no qualms about departing. I record what I did, along with a rationale for the departure; we track how the patient did with the new treatment; and if it seemed helpful we may add it to future versions of the manual. In my experience, having a clear sense of ‘what I usually do’, which mostly works quite well, helps me to spot unusual and difficult cases more quickly and frees up time for me to consider them.

All in all, I don’t think manuals are necessary for all conditions–for example, we all learned very well in school how to treat back pain, and ample evidence shows that ordinary acupuncture treatment is effective for it[10]. The effort of developing a manual is simply not needed there. For areas where we’re working closely with Western providers, however, where the patients are particularly vulnerable, or where the diagnosis and treatment planning are complex, I hope that researchers, teachers, and also practitioners who have reached a high degree of expertise, will consider evidence-informed, practice-based manualization as a vehicle for making that expertise available to more patients from more practitioners.

[1] Citkovitz, C., Klimenko, E., Bolyai, M., Applewhite, L., Julliard, K., & Weiner, Z. (2009). Effects of Acupuncture During Labor and Delivery in a US Hospital Setting: A Case–Control Pilot Study. The Journal of Alternative and Complementary Medicine, 15(5), 501-505.

[2] Citkovitz, C. (2015). Acupuncture during stroke rehabilitation: development of a manual for researching a complex intervention (Doctoral dissertation, University of Westminster).

[3] Saver, J. L., Fonarow, G. C., Smith, E. E., Reeves, M. J., Navalkele, D., Grotta, J. C., … & Schwamm, L. (2015). Shape of the TPA Time-Benefit Curve: Insights From The National US Get With The Guidelines-Stroke Population. Stroke, 46(Suppl 1), A50-A50.

[4] MacDorman, M. F., Matthews, T. J., Mohangoo, A. D., & Zeitlin, J. (2014). International comparisons of infant mortality and related factors: United States and Europe, 2010. National vital statistics reports: from the Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System, 63(5), 1-6.

[5] Furth, C., Zeitlin, J. T., & Hsiung, P. C. (2007). Thinking with cases: specialist knowledge in Chinese cultural history. University of Hawaii Press.

[6] Alban, J., Citkovitz, C., & Julliard, K. (2005). Assessment of Hospital Staff’s Attitudes Towards Acupuncture in an Inpatient Setting. Paper presented at the The Twelfth Annual Symposium of the Society for Acupuncture Research, Albuquerque, New Mexico.

[7] Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. Bmj, 337, a1655.

[8] Oleson, T. (2013). Auriculotherapy manual: Chinese and Western systems of ear acupuncture. Elsevier Health Sciences.

[9] “Responsive Manualization” of Acupuncture Therapy for Chronic Pain in Urban Primary Care Settings

Arya Nielsen Claudia Citkovitz Belinda J. Anderson Melissa McKee Benjamin Kligler

In: Kligler, B., Buonora, M., Gabison, J., McKee, D., Filshie, J., Minchom, A., … & Popat, S. (2016). Abstracts from the Society for Acupuncture Research 2015 International Conference Reaching Across Disciplines to Broaden the Acupuncture Research Network November 12–14, 2015 Boston, MA. The Journal of Alternative and Complementary Medicine, 22(1), A1-A46.

[10] Vickers, A. J., Cronin, A. M., Maschino, A. C., Lewith, G., MacPherson, H., Foster, N. E., … & Acupuncture Trialists’ Collaboration. (2012). Acupuncture for chronic pain: individual patient data meta-analysis. Archives of internal medicine, 172(19), 1444-1453.

[11] Timmermans, S., & Berg, M. (2010). The gold standard: The challenge of evidence-based medicine and standardization in health care. Temple University Press.

Featured Posts:

• How to Market Yourself and Your Practice
• Can Acupuncture Really Benefit Stroke Recovery?
• Supportive Care for Cancer Patients During and After Chemotherapy Treatments
• The 5 Phases of Event Training Using Sports Acupuncture
• Incorporating Aromatherapy into Your Practice