How Minor Tweaks in Scientific Methods Can Have a Major Impact
In our daily work with scientists from all over the world, we hear a lot of stories of how minor tweaks in scientific methods can have a major impact on someone’s research. This might have been the result of a publication lacking details that are needed to reproduce the findings, the method description could have had typos — stating 200 µl instead of 20 µl, or it simply did not mention a specific condition. Another reason could be that some specific details went missing over the years of a lab working with the method. For example, new lab members may not have been aware of the exact details, the reagent composition of a vendor could have changed, or some equipment in the lab had been replaced.
Our podcast, Minor Tweak, Major Impact, aims to highlight the importance of detailed method sharing for rigorous and reproducible science. The podcast looks into a series of stories related to following protocols and how a minor tweak can produce major results. In this blog, we are highlighting some of the most intriguing stories that we’ve had the pleasure of hearing of how a minor tweak, majorly impacted the research of others.
Making media from scratch
In the very first Minor Tweak, Major Impact episode, Luke Schwerdtfeger of Colorado State University, discusses a time when he had been using a protocol that his advisor originally developed in the 1980’s where they kept an entire cross-section of mouse or human intestines alive in a dish.
They were able to keep it alive for about a week at the time, however, one day he checked on his cultures at 24 hours and they were all dead. His advisor thought it was probably an infection, and told him to try it again. Once he tried again, and it didn’t work, he kept trying and that repeated itself for about two months of doing the experiment every week. About two months in, they heard from an old friend of his advisor who used the same media as they did but for brain slice cultures, and their cultures were dying as well. He asked them what they were doing to fix it, and they told him that they made their own media from scratch instead of buying it, which ended up working. They then tried to use their homemade media, and sure enough his cultured survived.
Buffer pH matters
Dr. Samantha Morris at Washington University recalls a time when she was a graduate student, and the method she was using didn’t work for months. Her PI then sent her to a collaborators lab in Japan, where she worked directly from the collaborator who had trained him. During this, she quickly realized that she was missing a lot of information back in Cambridge, where she was doing her PhD at the time. She recalls, “We didn’t have good reagents, I didn’t have good protocols, and I also realized that I hadn’t pHed my buffer properly.” She then states that once she returned from Japan, her experiments started working, and taught her a lot about using good protocols and having protocols that you can follow clearly.
Producing fibronectin from human fibroblasts
Dr. Kenneth Yamada of NIH told a story of when he was trying to produce substantial amounts of this newly described protein fibronectin from human fibroblasts. Over a span of about a year, they developed a very effective way of isolating plenty of this protein using procedures that continued to work well for multiple months. But then for some reason the production systems suddenly produced less and less fibronectin and it became untenable.They ended up changing the cell culture surfaces of the type of medium and serum extraction times, methods, cell density, and various other experimental parameters. But they were never able to produce substantial amounts of this protein through their extraction methods. It ultimately took them some time to find an alternative means of isolating the protein using a completely different approach involving serum free medium active secretion by fiberglass and so forth.
“But the key point here is that there can sometimes be cases in which we find that a procedure suddenly and unaccountably fails to work. And if we’re lucky, which we weren’t, you can identify the altered variable, but in other cases like ours, perhaps because we are working with a biological system, we may never find out what happened. I actually never did and had to switch to an alternative approach, but that’s the worst case and there are much better situations that can often occur in this type of reproducibility problem.”
Seasonal changes in water quality
Richard Harris, science correspondent at NPR and author of Rigor Mortis — a book about rigor and reproducibility in biomedical research, recalls a story from the University of Colorado at Boulder where a seasonal change in the water caused experiments to only work seasonally due to changes in water quality. They were able to get good results in the spring when the water was coming in, however in the winter they couldn’t.
“It was quite the anecdote to realize, wow just something as minor as, you know, a change in whatever water is running off the Rocky Mountains from one season to another can actually completely change the results of an experiment.”
The glycerol stock was slightly different
In our most recent episode, we spoke with Dr. Ross Kent about a story from a few years ago when he was at the University of Exeter working with thermophilic bacteria, the bacillus species that is a normal organism.
“We had a situation where we were working with a couple of collaborators and we’d been sent strains and we’d started working with them and getting transformation protocols up and running and it was going really well. And basically we had a situation where we’ve made these strains quite painstakingly, like transformed them and got this transformation protocol up and running. And then our wild type glycerol stock for that, you know, like the strain that we were engineering, everything into the glycerol stock was unfortunately left on a bench by someone who shall remain nameless. And we were like, okay, well that’s these things happen. We remade the glycerol. And then it was Christmas. We came back from Christmas and nothing worked. We couldn’t transform it. It was antibiotic resistant. We had no idea what was going on and it was very frustrating, very strange. We tried absolutely everything we could think of to try and get this protocol back working again. And at the time we could never work it out. So we ended up having to get that strain from a culture collection again, or possibly from a collaborator, I don’t remember exactly why, but we basically had to get this wild type strain back into the lab and we were then able to transform the bug again. And at the time we never really knew why.
I’d continue to work on this same organism for a year or two after this had happened. And what we kind of realized was that basically it was the way we’d made our glycerol stocks was slightly different. So instead of making the glycerol from an overnight culture, which we’d done when we replaced the stock previously, I think that they’d been made from a like mid experiential face culture because that’s how I later started to prepare them and it worked fine. So just that, that slight difference in timing and the, you know, in the morphology of the stock that we were using, which we were then, you know, growing overnight and restarting the cell cycle, if you’d like, but for whatever reason it did not work. And you think, oh, it’s just a glycerol stock, right? Like as long as it grows, you don’t really think about it.”
Do you have a Minor Tweak, Major Impact story that you would like to share? Let us know!
Referenced Links:
