After 6 weeks, I’ve finally finished my blinded study of melatonin to increase sleep duration. Here’s the write-up. Hope you find it interesting.

If you have any suggestions for other supplements or interventions for me to try, please let me know in the comments.

– QD

Summary:

Over the few months, I’ve been making an effort to get more sleep. I’ve been able to hit an average time asleep of ~7h and, qualitatively, I’ve been feeling a lot less tired and have been able to concentrate better in the afternoons.  

I’d like to see if sleeping even longer would result in further improvement, but have been unable to do so due to routinely waking up before my alarm. 

In an attempt to sleep longer, I decided to try melatonin. It’s typically used to control when you go to sleep, but it last long enough in the bloodstream that it might impact time asleep as well (Examine.com, ACX). Based on suggestions solicited from the ACX open thread, I ran a 28 day, blinded, randomized trial of 0.3 & 3 mg melatonin, both regular and extended release. 

Here’s the summary of the results (full details below):

• Measurement Reliability:
• Sleep measurements from my Apple Watch are occasionally off by several hours, sometimes demonstrably off by up to 8 min., and don’t correlate with manually recorded times asleep.
• For all subsequent analyses, I will only use manually recorded sleep data
• Measurement Effect:
• Contemporaneous recording of waking disrupted my sleep, leading to more recalled wake-ups and possibly increased fatigue
• For future sleep studies, I will record waking and other observations only upon arising or find an automated tracker that can record them without conscious attention on my part.
• Melatonin Effect:
• Melatonin had no observable effect on my sleep duration or any other metric examined.
• It’s possible that it had an effect that was too small to be observed using my experiment design. However, if that’s the case the effect is too small to be of interest/use to me.

While it’s disappointing that the melatonin didn’t have any effect on my sleep duration, I did learn a lot about how (and how not) to measure sleep. Based on these results, I’m going to keep manually recording how I slept when I wake up and see if I can identify any patterns I missed when previously looking only at data collected from my watch.

I’d also like to investigate other supplements reported to improve sleep duration & quality. Some recommendations I’ve gotten over the last few weeks include L-theanine, magnesium, and tryptophan.

Does anyone else have any suggestions for supplements or interventions I should try?

Details:

Purpose

• To determine if taking melatonin supplements in the evening extends the duration of my time asleep and subjective fatigue given a consistent bedtime.

• To determine if taking melatonin supplements and/or sleeping longer affects subjective fatigue, blood glucose, heart rate variability, or pulse

Background

Over the few months, I’ve been making an effort to get more sleep. I’ve been able to hit an average time asleep of ~7h and, qualitatively, I’ve been feeling a lot less tired and have been able to concentrate better in the afternoons.  

I wanted to see if sleeping even longer will result in further improvement. However, I’ve routinely been waking up before my morning alarm. I stay in bed (eyes closed) until the alarm, but can’t go back to sleep. Based on my data so far, there’s no clear correlation with time I fell asleep or total time asleep. 

I’d like to test some interventions to sleep longer. I already exercise in the evenings and for as long as I’m willing to do (~30 min. high intensity, 5-10 min. stretching), my last meal is 4h before going to bed, and my CGM does not show a consistent rise in blood sugar before waking up.

Given that, my next thought was to try melatonin. It’s typically used to control when you go to sleep, but it last long enough in the bloodstream that it might impact time asleep as well (Examine.com, ACX). Based on suggestions solicited from the ACX open thread, I ran a 28 day, blinded, randomized trial of 0.3 & 3 mg melatonin, both regular and extended release. 

Design/Methods

The original experimental design was pre-registered here & here. 

Materials

• Melatonin (I couldn’t find a single supplier for all types):
• 0.3 mg – Nootropics Depot Melatonin Capsules
• 3 mg – Nature Made Melatonin
• 0.3 mg, extended release – Life Extension Melatonin 6 Hour Time Released
• 3 mg, extended release – Douglas Laboratories Melatonin PR
• Gel Capsules: XPRS Nutra Size 000 Empty Capsules (Red/Black)

Blinding

• Melatonin was placed in opaque, size 000 gel capsules (6 per type).
• Dosages were randomly assigned to days using the excel random number generator and placed into a coded pill container by a second person (not me). 
• Data was unblinded after the completion of the experiment.

Procedure

• First 28 days:
• At 9pm, I took that days gel capsule with minimal water.
• At 9:30pm, I turned off the lights in by bedroom and attempted to go to sleep.
• Contemporaneously, I recorded:
• Times I woke up in the night
• Subjective tiredness (1-3 scale) upon arising and at 3p (I kept forgetting to do this, so after a few days, I gave up on it).
• If any other unusual events occur
• 13 day extension:
• At 9:30pm, I turned off the lights in by bedroom and attempted to go to sleep.
• When I woke up, I recorded:
• Time I woke up
• How many times I remembered waking up in the night
• If any other unusual events occur
• Exclusions:
• Two days were excluded from the experiment. 
• On 7/29/21 I had a late night event, well past my normal bed time, so I skipped the experiment entirely (did not take a pill and did not record sleep).
• On 8/4/21 I had a series of mishaps. I took the pill at the wrong time, my continuous glucose alarm erroneously triggered and woke me up, and I woke up a second time by rolling the wrong way and hurting my arm.

Measurements

• Sleep, HRV, & pulse: Apple Watch Series 5 + Autosleep app
• Sleep, times waking up during the night: manual recording both contemporaneously and after waking up (see Procedure, above)
• Glucose: Dexcom G6 CGM
• Blood pressure (weekly): Omron 10

Analyses

• Pre-registered:
• Sleep time, wake time, total time asleep, number of times waking up before alarm, fasting blood glucose, time of morning blood glucose rise, pulse, and HRV will be plotted vs. dose, release type, and recording method. 
• Note: since no significant effect on sleep parameters was found, I skipped the analysis of morning blood glucose rise, which is time-consuming.
• Additional exploratory analyses (not pre-registered):
• Time to first wake-up
• Comparison of time asleep as measured by Apple Watch vs. manual recording
• Student’s t-test was used to test if the values for any of the above metrics were different for each dose, release type, or combination of dose & release type vs. the placebo. p-values were calculated for each comparison and corrected for multiple-comparison
• Note: Since no comparison was statistically significant, I did not bother to correct for multiple comparisons.

Data Processing & Visualization

Autosleep data was exported and reformatted using a custom python script. Both Autosleep and manually recorded data was visualized using Tableau.

Data: here

Results & Discussion

Measurement Reliability

Before I can evaluate the effect of melatonin, I first need to look at the reliability of the measurements. I routinely check my continuous glucose monitor against an Abbot Freestyle Lite blood glucose (fingerstick) meter and know it to be reliable to within ~4 mg/dL, so no concern there. For pulse and HRV measurements, I don’t have any way to validate the data, so for purposes of this analysis, I’ll accept them as accurate.

That leaves the sleep measurements from my Apple Watch. Since this is the first time I’ve manually recorded when I went to sleep and woke up, I was very interested to see whether the automatically recorded sleep on my watch matched. Unfortunately, it didn’t. As can be seen in Figure 1 (top, left), several days exhibit large discrepancies between the manual and watch recorded time asleep. Inspecting the data, it’s clear that it’s the watch that is incorrect. In all cases where the watch reported a typical time asleep while manual recording showed time asleep was was low, the watch had recorded that I went to sleep when I first lied down and turned off the lights (9:30p), while I had manually recorded that I had insomnia and didn’t fall asleep until much later (only done when I never feel asleep in the first place). In all cases where the watch reported higher than normal time asleep, it had recorded that I was still sleeping hours past my alarm (which never happens) and on days when I know I was working or driving to work by 5:30a.

Even after removing the data points for which the watch is clearly incorrect, there’s still a relatively poor correlation between the manual and watch recorded data (Figure 1, top right), with an R2 of only 0.7. I don’t have a way to determine which is more accurate, but given the intermittent extremely inaccurate measurements by the watch and the fact that the watch shows waking times past my alarm when I know I was up and moving around, I will use the manually recorded data for all future analysis.

Conclusions:

• Sleep measurements from my Apple Watch are occasionally off by several hours, sometimes demonstrably off by up to 8 min., and don’t correlate with manually recorded times asleep.
• For all subsequent analyses, I will only use manually recorded sleep data

Measurement Effect

During the course of the experiment, I noticed that I was waking up in the middle of the night more often than I remembered doing before the study started. Based on this, I became concerned that the process of noting down when I woke up during the night was disrupting my rest and causing me to be more aware of it. 

To check this hypothesis, I extended the sleep study, taking no melatonin and only noting down the number of times I remember waking up after I finally get up for good in the morning. This is, of course, not blinded, but can at least test whether the initial observation was correct.

The results of this extension are show in Table 1 and Figure 2. As hypothesized, once I stopped contemporaneous recording of when I woke up, the number of recorded wakings dropped significantly (2.7 -> 1.9, p=0.2). My time in bed also dropped, (not statistically significant), driven by getting out of bed earlier. All this strongly suggests that the process of contemporaneous recording of waking was disrupting my sleep, making me more tired and enabling or inducing me to stay in bed longer.

Conclusions:

• Contemporaneous recording of waking disrupted my sleep, leading to more recalled wake-ups and possibly increased fatigue
• For future sleep studies, I will record waking and other observations only upon arising or find an automated tracker that can record them without conscious attention on my part.

Melatonin Effect

To assess the effect of melatonin on sleep duration, I looked at the following metrics:

• Time in bed/asleep
• # of times I woke up during the night (including final wake up)
• Time to 1st wake up
• Fasting blood glucose (the next day)
• Average blood glucose (the next day)
• Pulse (while sleeping)
• Heart rate variability (HRV)

For each metric, I used students t-test to assess whether the value for the placebo was different that for each dose, type (normal or extended release) and combination of dose & type. The data is show in the graphs and tables below. In no case was the p-value <0.05, indicating no significant different from the placebo. 

Since each dataset was relatively small (5-6 days per unique condition) and I had insomnia on three of the days of the experiment, I also tested whether any of the metrics became significantly different after excluding the days I had insomnia. The only one that did was pulse, which showed a small, but statistically significant increase for the 0.3 mg, normal release and aggregated normal release conditions. However, since this analysis was not pre-registered, I did not correct for multiple comparisons, my pulse has been trending downward during the course of the experiment, and there’s no dose dependence, this result is very likely due to chance.

Conclusions:

• Melatonin had no observable effect on my sleep duration or any other metric examined.
• It’s possible that it had an effect that was too small to be observed using my experiment design. However, if that’s the case the effect is too small to be of interest/use to me.

Conclusions & Next Experiments

• Measurement Reliability:
• Sleep measurements from my Apple Watch are occasionally off by several hours, sometimes demonstrably off by up to 8 min., and don’t correlate with manually recorded times asleep.
• For all subsequent analyses, I will only use manually recorded sleep data
• Measurement Effect:
• Contemporaneous recording of waking disrupted my sleep, leading to more recalled wake-ups and possibly increased fatigue
• For future sleep studies, I will record waking and other observations only upon arising or find an automated tracker that can record them without conscious attention on my part.
• Melatonin Effect:
• Melatonin had no observable effect on my sleep duration or any other metric examined.
• It’s possible that it had an effect that was too small to be observed using my experiment design. However, if that’s the case the effect is too small to be of interest/use to me.

While it’s disappointing that the melatonin didn’t have any effect on my sleep duration, I did learn a lot about how (and how not) to measure sleep. Based on these results, I’m going to keep manually recording how I slept when I wake up and see if I can identify any patterns I missed when previously looking only at data collected from my watch.

I’d also like to investigate other supplements reported to improve sleep duration & quality. Some recommendations I’ve gotten over the last few weeks include L-theanine, magnesium, and tryptophan.

Does anyone else have any suggestions for supplements or interventions I should try?

As always, please let me know if you have any thoughts or suggestions.

– QD