Want to know the secret to always running successful tests?
The answer is to formulate a hypothesis.
Now when I say it’s always successful, I’m not talking about always increasing your Key Performance Indicator (KPI). You can “lose” a test, but still be successful.
That sounds like an oxymoron, but it’s not. If you set up your test strategically, even if the test decreases your KPI, you gain a learning, which is a success! And, if you win, you simultaneously achieve a lift and a learning. Double win!
The way you ensure you have a strategic test that will produce a learning is by centering it around a strong hypothesis.
So, what is a hypothesis?
By definition, a hypothesis is a proposed statement made on the basis of limited evidence that can be proved or disproved and is used as a starting point for further investigation.
Let’s break that down:
It is a proposed statement.
- A hypothesis is not fact, and should not be argued as right or wrong until it is tested and proven one way or the other.
It is made on the basis of limited (but hopefully some) evidence.
- Your hypothesis should be informed by as much knowledge as you have. This should include data that you have gathered, any research you have done, and the analysis of the current problems you have performed.
It can be proved or disproved.
- A hypothesis pretty much says, “I think by making this change, it will cause this effect.” So, based on your results, you should be able to say “this is true” or “this is false.”
It is used as a starting point for further investigation.
- The key word here is starting point. Your hypothesis should be formed and agreed upon before you make any wireframes or designs as it is what guides the design of your test. It helps you focus on what elements to change, how to change them, and which to leave alone.
How do I write a hypothesis?
The structure of your basic hypothesis follows a CHANGE: EFFECT framework.
While this is a truly scientific and testable template, it is very open-ended. Even though this hypothesis, “Changing an English headline into a Spanish headline will increase clickthrough rate,” is perfectly valid and testable, if your visitors are English-speaking, it probably doesn’t make much sense.
So now the question is …
How do I write a GOOD hypothesis?
To quote my boss Tony Doty, “This isn’t Mad Libs.”
We can’t just start plugging in nouns and verbs and conclude that we have a good hypothesis. Your hypothesis needs to be backed by a strategy. And, your strategy needs to be rooted in a solution to a problem.
So, a more complete version of the above template would be something like this:
In order to have a good hypothesis, you don’t necessarily have to follow this exact sentence structure, as long as it is centered around three main things:
- Presumed problem
- Proposed solution
- Anticipated result
After you’ve completed your analysis and research, identify the problem that you will address. While we need to be very clear about what we think the problem is, you should leave it out of the hypothesis since it is harder to prove or disprove. You may want to come up with both a problem statement and a hypothesis.
Problem Statement: “The lead generation form is too long, causing unnecessary friction.”
Hypothesis: “By changing the amount of form fields from 20 to 10, we will increase number of leads.”
When you are thinking about the solution you want to implement, you need to think about the psychology of the customer. What psychological impact is your proposed problem causing in the mind of the customer?
For example, if your proposed problem is “There is a lack of clarity in the sign-up process,” the psychological impact may be that the user is confused.
Now think about what solution is going to address the problem in the customer’s mind. If they are confused, we need to explain something better, or provide them with more information. For this example, we will say our proposed solution is to “Add a progress bar to the sign-up process.” This leads straight into the anticipated result.
If we reduce the confusion in the visitor’s mind (psychological impact) by adding the progress bar, what do we foresee to be the result? We are anticipating that it would be more people completing the sign-up process. Your proposed solution and your KPI need to be directly correlated.
Note: Some people will include the psychological impact in their hypothesis. This isn’t necessarily wrong, but we do have to be careful with assumptions. If we say that the effect will be “Reduced confusion and therefore increase in conversion rate,” we are assuming the reduced confusion is what made the impact. While this may be correct, it is not measureable and it is hard to prove or disprove.
To summarize, your hypothesis should follow a structure of: “If I change this, it will have this effect,” but should always be informed by an analysis of the problems and rooted in the solution you deemed appropriate.
A/B Testing 101: How to get real results from optimization
The True Value of Data
15 Years of Marketing Research in 11 Minutes
Marketing Analytics: 6 simple steps for interpreting your data
Website A/B Testing: 4 tips to beat an unbeatable landing page
Intra-individual variation allows an explicit test of the hygric hypothesis for discontinuous gas exchange in insects
Caroline M. Williams,1Shannon L. Pelini,2,†Jessica J. Hellmann,2 and Brent J. Sinclair1,*
1Department of Biology, University of Western Ontario, London, ON N6A 5B7, Canada
2Department of Biological Sciences, University of Notre Dame, Notre Dame, IN 46556, USA
*Author for correspondence (ac.owu@7alcnisb).
†Present address: Harvard Forest, Harvard University, Petersham, MA 01366, USA
Author information ►Article notes ►Copyright and License information ►
Received 2009 Oct 5; Accepted 2009 Oct 23.
Copyright © 2009 The Royal Society
This article has been cited by other articles in PMC.
The hygric hypothesis postulates that insect discontinuous gas exchange cycles (DGCs) are an adaptation that reduces respiratory water loss (RWL), but evidence is lacking for reduction of water loss by insects expressing DGCs under normal ecological conditions. Larvae of Erynnis propertius (Lepidoptera: Hesperiidae) naturally switch between DGCs and continuous gas exchange (CGE), allowing flow-through respirometry comparisons of water loss between the two modes. Water loss was lower during DGCs than CGE, both between individuals using different patterns and within individuals using both patterns. The hygric cost of gas exchange (water loss associated with carbon dioxide release) and the contribution of respiratory to total water loss were lower during DGCs. Metabolic rate did not differ between DGCs and CGE. Thus, DGCs reduce RWL in E. propertius, which is consistent with the suggestion that water loss reduction could account for the evolutionary origin and/or maintenance of DGCs in insects.
Keywords: discontinuous gas exchange, respiratory water loss, Lepidoptera
Discontinuous gas exchange cycles (DGCs) have evolved independently at least five times in insects (Marais et al. 2005). The evolutionary pressures that lead to DGCs are debated (Chown et al. 2006). DGCs consist of three phases: closed phase during which spiracles are closed and there is no external gas exchange; flutter phase where spiracles rapidly open and close, allowing bulk inflow of air, and open phase where spiracles are open to allow unrestricted gas exchange (Chown et al. 2006).
Three main adaptive hypotheses have been proposed to explain the origin and maintenance of DGCs (Chown et al. 2006). The hygric hypothesis contends that DGCs have evolved to limit respiratory water loss (RWL) by maximizing the time that the spiracles are closed, and minimizing water efflux due to bulk inward convection in the F-phase (Chown et al. 2006). The chthonic–hygric hypothesis (Lighton & Berrigan 1995) states that DGCs originated in insects inhabiting hypoxic and hypercapnic (primarily underground) environments to increase O2 and CO2 diffusion gradients, with coincidental water savings. The oxidative damage hypothesis (Hetz & Bradley 2005) suggests that DGCs minimize oxidative damage during periods of low metabolic demand, by maintaining low tracheal PO2 while retaining delivery capacity during periods of high metabolic demand (e.g. flight).
Here, we focus on the water retention benefits of DGCs, primarily addressing the hygric hypothesis. We note the difficulty in distinguishing the hygric and chthonic–hygric hypotheses based on water loss, but the hygric hypothesis may be rejected independently of the chthonic and oxidative damage hypotheses since CO2 and O2 partial pressures are central to the latter (Chown et al. 2006). The hygric hypothesis predicts that (i) water lost per CO2 released will be lower for insects using DGCs (see also Lighton & Turner 2008) and (ii) DGCs will decrease RWL.
Measurement of water loss within DGCs shows that RWL is greater when the spiracles are open (see Chown 2002). DGCs are longer in species from xeric environments (White et al. 2007), while cyclic and continuous patterns are more prevalent in mesic habitats (Marais et al. 2005). RWL was lower in individual ants that did not express DGCs; however, those individuals also had lower metabolic rates (Gibbs & Johnson 2004). Manipulation of environmental variables can force insects to abandon DGCs (e.g. Lighton & Turner 2008; Terblanche et al. 2008), but to our knowledge there have been no comparisons of RWL in individuals that use both DGCs and continuous gas exchange (CGE) under ecologically relevant conditions.
Erynnis propertius (Lepidoptera: Hesperiidae) overwinter as quiescent late-instar larvae in rolls of dry oak leaves (Prior et al. 2009). Quiescent larvae probably experience desiccation during the overwintering period as no feeding occurs. Under benign conditions, individuals use both DGCs and CGE, allowing a direct comparison of water loss rates both between and within individuals during DGCs and CGE. From the hygric hypothesis, we expect the ratio of water loss to CO2 emission to be lower during DGCs than during CGE, and that the contribution of RWL to total water loss will be lower during DGCs. Support for these predictions under benign environmental conditions is a prerequisite for comprehending the role of water loss in the evolution of DGCs.
2. Material and methods
Erynnis propertius larvae were reared from adults caught in spring 2007 from Oregon, USA, and British Columbia, Canada. Larvae were fed fresh Garry oak (Quercus garryana) leaves until the sixth instar when they became quiescent (Pelini et al. 2009). Then, larvae were housed in Sanyo MIR-153 incubators (Sanyo Scientific, Bensenville, IL) in 25 ml plastic containers on moist vermiculite without food, at 8 : 1°C (day : night) and a 13D : 11L photoperiod. Total water content was determined gravimetrically for nine individuals that were not used in respirometry.
Volume of water and CO2 released per unit time (H2O, CO2) by E. propertius larvae (n = 39) were measured for 4 h at 8°C after 3 h acclimation using flow-through respirometry (Lighton 2008; details in electronic supplementary material). Each larva was measured once, at a randomly assigned time between 8.00 and 20.00. All comparisons were made between complete DGCs and 68 min blocks of CGE, the latter chosen to match the mean cycle time of the DGCs (see electronic supplementary material for details of data selection).
Mean ± s.e.m. is reported throughout. All statistical analyses were performed in R (R Project v. 2.8.1; www.r-project.org/). Where ratios or percentages are presented, statistical analyses were performed on raw data using analysis of covariance (ANCOVA). CO2 and H2O (µl h−1) were compared between modes using repeated measures ANCOVA (individuals using mixed patterns) or ANCOVA (effect of mode between individuals) with the covariates mass and time. H2O (µl h−1) between individuals was also compared using an ANCOVA with the covariates mass and time, and CO2 to determine whether the molar ratios were significantly different between groups. CO2 and H2O were log10-transformed prior to this analysis. To determine the hygric cost of gas exchange (Gibbs & Johnson 2004), H2O was regressed against CO2 and the resulting slope used to estimate the incremental increase in water loss associated with CO2 release (electronic supplementary material, figure S1). H2O/CO2 slopes were compared between continuous and discontinuous gas exchange with a t-test (between individuals) or paired t-test (within individuals). Cuticular water loss for all individuals and modes was estimated as the intercept of the H2O/CO2 regression (Gibbs & Johnson 2004) and compared between CGE and DGCs using an ANCOVA with total water loss as a covariate. RWL was calculated by subtracting cuticular from total water loss and compared between CGE and DGCs using an ANCOVA with cuticular water loss as a covariate.
No movement was detected in any larvae during respirometry. Fifteen individuals used solely CGE, 18 individuals used solely DGCs and six individuals switched between patterns during the course of one measurement period (figure 1). In those that switched between patterns, four of six switched from CGE to DGCs, with one switching from DGCs to CGE, and a sixth switching from CGE to DGCs and back again. Water loss declined during a respirometry run for both modes of gas exchange (F2,12 = 22.72, p < 0.001), while CO2 did not (F2,12 = 1.17, p = 0.34). Total water content of n = 9 caterpillars was 3.08 ± 0.4 g H2O g(dry mass)−1. CO2 did not differ between gas exchange patterns either within or between individuals (within: F1,3 = 2.64, p = 0.20; between: F1,29 = 2.61, p = 0.12; table 1). Time and mass were not statistically significant covariates of CO2 in either analysis (p > 0.1).
Example of CO2 (grey lines) and H2O (black lines) emission traces from larvae of E. propertius: (a) solely CGE; (b) solely DGCs and (c) a mixture of patterns.
Gas exchange and water loss parameters in E. propertius larvae using both or either mode of gas exchange. Mean ± s.e.m. presented.
Water loss was significantly lower during DGCs than during CGE both within and between individuals (within: F1,3 = 34.75, p = 0.010; between: F1,28 = 5.59, p = 0.025; figure 2, table 1). Time was not a statistically significant covariate (p > 0.1) for H2O either between or within individuals, nor was mass within individuals (F1,3 = 2.64, p = 0.20). However, mass was positively correlated with H2O between individuals (F1,28 = 15.06, p < 0.001). The ratio of H2O to CO2 was higher during CGE between individuals (F1,29 = 1.84, p = 0.02; table 1). The slopes of the regression of H2O on CO2 were higher during CGE than during DGCs between but not within individuals, although the trend was in the same direction (between: t14 = 2.59, p = 0.020; within: t5 = 1.11, p < 0.1; table 1). RWL accounted for significantly more of the total water loss during CGE both between and within individuals (between: F1,29 = 5.41, p = 0.027; within: F1,3 = 22.77, p = 0.017; table 1). Cuticular water loss did not differ between DGCs and CGE either between (F1,29 = 1.68, p = 0.206) or within (F1,3 = 1.69, p = 0.284) individuals.
Water loss during continuous compared with discontinuous gas exchange in E. propertius individuals that used both modes. The line indicates equal water loss in both modes.
To our knowledge, this is the first time the hygric hypothesis of DGCs has been tested in a species where individuals exhibit both modes of gas exchange with comparable metabolic rates and without differential water balance status (e.g. Hadley & Quinlan 1993). Water loss in E. propertius is higher during CGE, both within individuals that use both patterns and between individuals exhibiting one or other mode. Thus, in this species, a DGC appears to confer a significant water conservation benefit. This contrasts with experiments where the mode of gas exchange or metabolic rate is manipulated (e.g. Lighton & Turner 2008; Terblanche et al. 2008; Contreras & Bradley 2009; Schimpf et al. 2009) and suggests that water conservation is an advantage that could lead to the origin or maintenance of DGCs in insects.
Grasshoppers abandoned DGCs when stressed by desiccation (Hadley & Quinlan 1993); in contrast, only two of six individuals that switched went from DGCs to CGE as they lost water in our study. Only six of the 39 caterpillars we observed switched gas exchange modes. We hypothesize that this results from the short (4 h) observation period, and that longer recordings would reveal a greater incidence of switching.
Between individuals, the slope of a regression of H2O on CO2 is higher during CGE, which indicates a reduced hygric cost of gas exchange during DGCs in E. propertius. In contrast, the H2O/CO2 slope did not differ among queens of Pogonomyrmex barbatus under different modes of gas exchange, but a DGC was abandoned at higher metabolic rates (Gibbs & Johnson 2004). We found no difference in CO2 between E. propertius individuals using DGCs and CGE. This challenges the oxidative damage hypothesis, which predicts that metabolic rate should determine the mode of gas exchange used by an individual through its influence on tracheal PO2 (Contreras & Bradley 2009).
Clearly, DGCs decrease RWL compared with CGE, while cuticular water loss does not differ. The relative contribution of RWL during DGCs (4.4%) in E. propertius is at the low end for tracheate arthropods, but RWL during CGE (12.8%) is consistent with other species (Chown 2002; Lighton et al. 2004). H2O declined throughout the experiment for both patterns of gas exchange which may indicate a steady decline in cuticular water loss, but is more likely due to water adhering to the inside of the plastic tubing. Baseline correction accounts for this effect (Lighton 2008). The fitness benefit of modulating RWL has been questioned (Chown 2002). However, caterpillars would take 19 days at 0% RH to reach an injurious 30 per cent reduction in water content if using DGCs, compared with only 10.6 days using CGE (see electronic supplementary material). Furthermore, insects adapted to xeric environments typically have reduced cuticular water loss (Chown 2002), increasing the benefit of reduced RWL. However, whether the RWL reduction during DGCs confers a fitness advantage requires further investigation.
We are grateful to John Terblanche and Jill Mueller for comments on an earlier draft and to John Lighton, Alex Kaiser and Barbara Joos for discussion and advice. This work was funded by NSERC and Canadian Foundation for Innovation (B.J.S.) and by US Department of Energy DE-FG02–05ER (J.J.H.). We thank landowners for allowing butterfly collections.
- Chown S. L. 2002Respiratory water loss in insects. Comp. Biochem. Physiol. A133, 791–804 (doi:10.1016/S1095-6433(02)00200-3) [PubMed]
- Chown S. L., Gibbs A. G., Hetz S. K., Klok C. J., Lighton J. R. B., Marais E. 2006Discontinuous gas exchange in insects: a clarification of hypotheses and approaches. Physiol. Biochem. Zool.79, 333–343 (doi:10.1086/499992) [PubMed]
- Contreras H. L., Bradley T. J. 2009Metabolic rate controls respiratory pattern in insects. J. Exp. Biol.212, 424–428 (doi:10.1242/jeb.024091) [PubMed]
- Gibbs A. G., Johnson R. A. 2004The role of discontinuous gas exchange in insects: the chthonic hypothesis does not hold water. J. Exp. Biol.207, 3477–3842 (doi:10.1242/jeb.01168) [PubMed]
- Hadley N. F., Quinlan M. C. 1993Discontinuous carbon dioxide release in the eastern lubber grasshopper Romalea guttata and its effect on respiratory transpiration. J. Exp. Biol.177, 169–180
- Hetz S. K., Bradley T. J. 2005Insects breathe discontinuously to avoid oxygen toxicity. Nature433, 516–519 (doi:10.1038/nature03106) [PubMed]
- Lighton J. R. B. 2008Measuring metabolic rates: a manual for scientists. New York, NY: Oxford University Press
- Lighton J. R. B., Berrigan D. 1995Questioning paradigms: caste-specific ventilation in harvester ants, Messor pergandei and M. julianus. J. Exp. Biol.198, 521–530 [PubMed]
- Lighton J. R. B., Turner R. J. 2008The hygric hypothesis does not hold water: abolition of discontinuous gas exchange cycles does not affect water loss in the ant Camponotus vicinus. J. Exp. Biol.211, 563–567 (doi:10.1242/jeb.010041) [PubMed]
- Lighton J. R. B., Schilman P. E., Holway D. A. 2004The hyperoxic switch: assessing respiratory water loss rates in tracheate arthropods with continuous gas exchange. J. Exp. Biol.207, 4463–4471 (doi:10.1242/jeb.01284) [PubMed]
- Marais E., Klok C. J., Terblanche J. S., Chown S. L. 2005Insect gas exchange patterns: a phylogenetic perspective. J. Exp. Biol.208, 4495–4507 (doi:10.1242/jeb.01928) [PubMed]
- Pelini S. L., Dzurisin J. D. K., Prior K. M., Williams C. M., Marsico T. D., Sinclair B. J., Hellmann J. J. 2009Translocation experiments with butterflies reveal limits to enhancement of poleward populations under climate change. Proc. Natl Acad. Sci.106, 11 160–11 165 (doi:10.1073/pnas.0900284106) [PMC free article][PubMed]
- Prior K. M., Dzurisin J. D. K., Pelini S. L., Hellmann J. J. 2009Biology of larvae and adults of Erynnis propertius at the northern edge of its range. Can. Entomol.141, 161–171
- Schimpf N. G., Matthews P. G. D., Wilson R. S., White C. R. 2009Cockroaches breathe discontinuously to reduce respiratory water loss. J. Exp. Biol.212, 2773–2780 (doi:10.1242/jeb.031310) [PubMed]
- Terblanche J. S., Marais E., Hetz S. K., Chown S. L. 2008Control of discontinuous gas exchange in Samia cynthia: effects of atmospheric oxygen, carbon dioxide and moisture. J. Exp. Biol.211, 3272–3280 (doi:10.1242/jeb.022467) [PubMed]
- White C. R., Blackburn T. M., Terblanche J. S., Marais E., Gibernau M., Chown S. L. 2007Evolutionary responses of discontinuous gas exchange in insects. Proc. Natl Acad. Sci.104, 8357–8361 (doi:10.1073/pnas.0608968104) [PMC free article][PubMed]
Articles from Biology Letters are provided here courtesy of The Royal Society