Recording live components¶
A live slide needs its Python process. A web page on GitHub Pages has none, and neither has a viewer who opens your deck
without nimbledeck run. A recording fixes that: you film the real application once, on your machine, and the slide plays
the film wherever the process is not there. When the process is there, the slide is live as before.
Record in three commands¶
npm install # once; playwright-chromium is already a dependency of a deck
brew install ffmpeg # once; any ffmpeg with libvpx-vp9 and libx264 (see "What you need")
nimbledeck record my-deck.md # starts the deck and its processes, films every live slide
git add public/recordings # the files are part of the deck
nimbledeck record starts the deck and its Python processes the way nimbledeck run does, opens the deck in Chrome, goes
to each slide with a <LiveCode>, <PyStream> or <Demo>, waits until the component is live, films it, and stops
everything. Options:
| Option | Default | Meaning |
|---|---|---|
--slide 11 19 |
all slides with a live component | film only these slides |
--seconds 8 |
8 | length of a clip (1 to 60). A seconds in the config wins |
--width 2560 |
2560 | width a whole slide would have in the video (480 to 2560). 2560 keeps every pixel; a smaller value scales down, only for a smaller file |
--out public/recordings |
<publicDir>/recordings |
where the files go |
--url http://localhost:3030 |
none | film a deck that is already running (with its processes) instead of starting one |
--fps 60 |
60 | frames per second of the clip (24 to 120) |
--realtime |
off | do not drive time: film every clip as it plays on the screen (see "Lockstep") |
--timeout 900 |
900 | wall-clock limit of one clip, in seconds (30 to 7200). A clip that goes past it stops with an error |
On a machine where another deck uses the default ports, set NIMBLEDECK_PORT_OFFSET as for run.
What you get, for a component called stream-nbody:
public/recordings/stream-nbody.webm VP9, the smaller file; Chrome, Firefox, Edge, and Safari where it plays WebM
public/recordings/stream-nbody.mp4 H.264 High, for Safari and iOS
public/recordings/stream-nbody.jpg a still: the last frame, at the full pixel size of the video
public/recordings/index.json the list of recordings: id, kind, seconds, size, files
The video is the box of the component, not the whole slide, at twice the pixels it has on a 1280 x 720 slide: a box of
1140 x 460 on the slide is a 2280 x 920 video. The page lays it out at 1140 x 460, so it is sharp on a 2x screen and
merely fine on a 1x one. The mouse pointer, Slidev's controls and toasts are not in it. In index.json, width and
height are the logical size (CSS pixels), pixelWidth and pixelHeight the pixels of the file.
What you need¶
An ffmpeg on your PATH built with libvpx-vp9 and libx264 (brew install ffmpeg, apt install ffmpeg, or the
Windows builds of gyan.dev). nimbledeck record asks it (ffmpeg -encoders) and writes what it can. Without one, it
falls back on the ffmpeg that Playwright downloads, which writes VP8 only, at a fixed quality and without an MP4: the
clips play, but they are softer, and it says so. The ladder in this page assumes the system ffmpeg.
Names¶
| Component | Recording name |
|---|---|
<PyStream name="nbody" /> |
stream-nbody |
<Demo name="compute" /> |
demo-compute |
<LiveCode id="plot"> |
live-plot |
<LiveCode> (no id) |
live-<slide number>, for example live-19 |
The same stream or demo on several slides is filmed once, on the first of them, and every slide plays that file. Give each
<LiveCode> an id when a slide holds two of them, or when you may reorder slides: the number in the default name is the
slide number at the time of recording.
Scripted steps¶
By default a recording shows the component as it starts. To show the point, describe what to do in
nimbledeck.config.json, under recordings:
{
"recordings": {
"stream-nbody": {
"seconds": 9,
"steps": [{ "wait": 1500 }, { "set": "gravity", "value": 2.5 }, { "wait": 1500 }, { "set": "particles", "value": 1200 }]
},
"live-19": {
"seconds": 12,
"steps": [{ "wait": 1200 }, { "type": "\nplt.plot(x, np.sin(2 * x))\nplt.grid(True)\n" }, { "wait": 1500 }]
},
"live-20": {
"steps": [{ "type": "print(sum(range(10)))", "replace": true }, { "key": "ControlOrMeta+Enter" }, { "wait": 1500 }]
}
}
}
Steps run in order from the first frame. Each has one action:
| Step | What it does |
|---|---|
{ "wait": 800 } |
Pauses for that many milliseconds. |
{ "type": "text" } |
Types into the <LiveCode> editor, at the end of the code. "replace": true clears the editor first. Keys come one at a time with uneven pauses, longer at spaces, punctuation and line ends, and the editor re-runs after each pause, so the plot changes while you watch. The same text gives the same rhythm every time. Only for live- recordings. |
{ "key": "ControlOrMeta+Enter" } |
Presses a key or a combination. ControlOrMeta is Cmd on a Mac and Ctrl elsewhere (Mod is accepted as a short form). |
{ "set": "gravity", "value": 2.5 } |
Moves a slider to a value. The slider is found by the text of its label. It moves in small steps, like a drag. Works for the sliders of <PyStream controls> and for marimo sliders. |
{ "click": "Run" } |
Clicks a button by its text. |
The clip lasts seconds, or the length of the steps plus a short tail when they take longer. nimbledeck record checks
the config before it starts and says which step is wrong. The editor indents by itself after a line break and closes
brackets, so type code that does not depend on the indentation you would get by hand, or keep it flat.
Lockstep: exactly 60 frames a second, whatever the machine¶
Filming a screen as it plays gives the frames the machine managed: 4 to 50 a second at 2x pixels, so clips repeat frames and
moving particles look choppy. For a <PyStream> the recorder therefore does not wait for time, it drives it. Each tick is 1/60
second of virtual time followed by one lossless screenshot of the component box, so a 9 second clip is exactly 540 frames
however slow the machine is (it takes longer to film, not to play). Streams are filmed this way by default; the options and
config key realtime turn it off.
What is lockstepped, and what is not:
| Clip | Mode |
|---|---|
stream-* (<PyStream>) |
lockstep, 60 fps exactly, the same frames on every recording |
live-* (<LiveCode>) |
real time: it waits for a Python runner that answers at its own speed. Frames are held to 60 fps. The manifest says lockstep: false and why |
demo-* (marimo) |
real time: the demo runs in a cross-origin frame in its own process, where the recorder cannot hold time. Output that changes in steps is filmed after each step settles. lockstep: false in the manifest |
How a stream is filmed:
- Chrome's
Emulation.setVirtualTimePolicygives each tick a budget of 1/60 s.setTimeout,setInterval,performance.nowandDate.nowfollow it exactly.requestAnimationFrameand CSS animations do not (Chrome drives them from real frames), so a small page script (PAGE_SHIMinpackages/cli/src/lockstep.mjs) runs queued rAF callbacks once per tick and sets every CSS animation, transition and Web Animation to the virtual time. Verified on a test page: 60 ticks give 60 rAF calls, 1000.0 ms ofperformance.nowand exactly one second of animation. - The stream protocol: the page sends
{"lockstep": true, ...parameters}, the server answers with the textlockstepand from then on computes exactly one frame per{"tick": n}message (seedocs/WRITING.md). The page holds the recorder's barrier (window.__ndLockstep.hold()) until the frame has arrived and been drawn, so no screenshot is taken half-way. The servers seed their random numbers, so a re-recording is the same clip. Slider steps are applied between ticks. - The frames go to ffmpeg as they come, into a lossless intermediate file; VP9 (two passes) and H.264 are then encoded from it.
- Chrome is started with
--disable-new-content-rendering-timeoutand a few compositor flags. Without them a screenshot taken while virtual time is paused can wait for a timeout that only virtual time could end, and never answers (it happened as soon as the deck mounted its other slides). Every wait of the recorder has a time limit and names what it was waiting for.
Not covered, and why: Web Audio and <video> elements follow the real clock, not virtual time (the clips have no sound anyway);
anything that talks to a process in real time is a stream (covered, by the tick protocol) or not covered. A stream server that
does not know the tick protocol makes the recorder stop with "the process did not answer {lockstep: true}". Deterministic
frames need deterministic servers: the three example streams are seeded.
Emulation virtual time was chosen over HeadlessExperimental.beginFrame, which would drive the compositor itself: on macOS
Chrome answers "BeginFrameControl is not supported on MacOS yet" (measured on Chrome 154 and the headless shell 153).
Timing on a laptop: 540 frames of a 2200 x 1000 box took about 40 s to film and a few minutes to encode. The manifest has fps,
frames and lockstep for every clip.
isolate: a stream whose slide has a paragraph or equations laid over it would film them too. "isolate": true in the clip's
config hides everything on the slide except the component and what it sits in, so the clip holds only the stream.
How playback works¶
<LiveCode>, <PyStream> and <Demo> look for recordings/<name>.webm or .mp4 when their process cannot be reached. The
browser is asked (canPlayType) about VP9 WebM and H.264 High MP4: the file it is sure about comes first, and WebM before MP4
when it is sure about both or neither (Safari answers "probably" for MP4 only, so it gets the MP4). The path starts at the deck's base, so a deck published under
/nimbledeck/demos/x/ finds /nimbledeck/demos/x/recordings/<name>.webm. If a recording exists, the component shows it
instead of the offline panel:
- a muted, looping, inline video that fills the box of the component (the box already has the logical size and shape of the recording, so nothing moves when the video loads), with the still as its poster. It loads when the slide shows;
- a small pill, "Recording of a live run", and a line, "Run it locally to make it live". Neither links anywhere;
- the live component's own controls are hidden, because the video already shows them, and its editor is inert;
- with the system setting "reduce motion", only the still is shown, with the same pill.
The recording appears after the process has been missing for 0.7 seconds, so the short moment of connecting never flashes a video. When the process comes back, the live component replaces it. Nothing is fetched from the network but your own files.
To use another file, give the component a path without extension, relative to the public folder:
<Demo name="compute" recording="/recordings/compute-slow" /> (.webm or .mp4 is added). A recording given this way is
looked up as written, even when the manifest does not list it.
The web edition¶
The build that publishes a deck on a static host (website/build.mjs, examples/showcase/build-web.mjs) leaves out the
slides that need nimbledeck run, except those whose live components all have a recording in public/recordings. Those
stay, and play the recordings. A slide with one live component and no recording is still left out.
Sizes¶
The clips of the example decks, 9 to 12 seconds (the sizes below are from before the stream clips went to 60 fps; see the manifest for the current ones), pixels as written in the file:
| Clip | Pixels | WebM (VP9) | MP4 (H.264) | Still |
|---|---|---|---|---|
stream-nbody (700 moving particles, 60 fps lockstep) |
2200 x 1000 | 3.1 MB | 4.1 MB | 234 KB |
stream-swarm (60 fps lockstep) |
2040 x 1120 | 3.1 MB | 4.0 MB | 221 KB |
live-19 |
2280 x 920 | 0.6 MB | 0.6 MB | 175 KB |
demo-compute, live-20 |
2280 x 920 | 0.2 to 0.3 MB | 0.2 to 0.3 MB | 47 to 81 KB |
For scale, ten clips of an earlier science deck (three real-time live-code clips of 1 to 2 MB, the streams 2 to 7 MB each) took
69 MB with their stills, both formats; the current science deck has no recordings at all, because it runs in the browser. Moving particles are the worst case, since every frame differs, so their
rate is capped (VP9 at 2.8 Mbit/s, H.264 at 3.4 Mbit/s, about 3 to 4 MB per 9 seconds); a calm slide never reaches the cap
and stays under 1 MB. A clip of 12 seconds with moving particles would reach 4 to 5 MB. Keep clips short: 8 to 12 seconds
loops well. --width 1280 makes a clip about a third of the size, at the 1x look of the first version.
How many frames the browser delivers depends on the machine: 35 to 60 a second for a calm page, 16 to 28 for particles drawn
on a busy laptop, and nothing while the screen does not change. The timeline repeats the last frame to fill 30 a second,
and nimbledeck record prints what it got (browser sent 28 frames/s).
What is not recorded¶
- Sound. The video has no audio track.
- The mouse pointer, and anything outside the box of the component, such as the slide title and footer.
- Anything that depends on the viewer: a recording does not react to clicks. Sliders and the editor on the poster or the video do nothing. It is a film, and it says so.
- Components that need no process (Orbit, Scene3D, Chart): they run in the browser anyway.
- Marimo's own drag handles: sliders are moved with the arrow keys, not by dragging.
Troubleshooting¶
never went live: the process did not start in 45 seconds. Runnimbledeck runon the deck and read.nimbledeck/logs/.no ffmpeg found: neither a system ffmpeg nor the Playwright one. Install ffmpeg, or runnpx playwright-chromium install ffmpeg.no system ffmpeg with VP9: using the ffmpeg of Playwright: the clips are VP8 and no MP4 is written. Install a full ffmpeg.- No
.mp4next to the.webm: the ffmpeg on your PATH has nolibx264. - The recording is slow (a few minutes for ten clips): VP9 runs in two passes on a 2560 x 1440 page, and a busy machine makes the browser deliver fewer frames. Close other heavy programs while it films.
stream-x is on slides ...ortwo components called live-...: give each<LiveCode>its ownid.
How it works¶
Chrome lays the deck out at 2560 x 1440, twice a 1280 x 720 slide, so text, canvases and the matplotlib figures of
<LiveCode> (drawn at the dpi of their on-screen size) are rendered natively at 2x, not scaled up. The recorder takes
the frames Chrome produces (the screencast of the Chrome DevTools protocol, as lossless PNG), keeps the latest one of
every 30th of a second, and holds the last frame on a still screen. It crops the box of the component and encodes it
without scaling.
Why a 2560 x 1440 page and not a 1280 x 720 one at a device scale factor of 2: the screencast always sends frames in CSS
pixels, so a 2x page would arrive at 1280 x 720. Page.captureScreenshot does deliver 2x, but at 16 to 25 frames a second
for a clip box, against 35 to 60 for the screencast. The pixels are the same, because Slidev scales the slide to the page.
The quality ladder, in QUALITY of packages/cli/src/record.mjs:
| Output | Encoder | Settings |
|---|---|---|
.webm |
libvpx-vp9 |
-crf 29 -b:v 0, two passes, -row-mt 1 -tile-columns 2, -deadline good -cpu-used 1, -tune-content screen, -g 60 (a keyframe every 2 s), yuv420p |
.mp4 |
libx264 |
-preset slow -crf 21 -profile:v high, yuv420p, -g 60, +faststart |
.webm without a system ffmpeg |
libvpx (VP8) |
-crf 10 -b:v 6M |
Both are 30 frames a second, RGB converted to BT.709 limited range and tagged as such (an untagged video shifts colours in some browsers), no audio. A lower CRF is sharper and bigger: 24 for VP9 or 18 for H.264 was indistinguishable from 29 and 21 on every clip, and the noisiest clip (1200 moving particles) grew by a quarter. The poster is cut from the last frame by Chrome itself, as a JPEG at quality 93.
Other choices measured and left out: playing the clips as 4:4:4 (yuv444p, VP9 profile 1) does not decode in Safari or
hardware decoders; AV1 (libsvtav1) is not written: Safari decodes it only on recent chips (A17 and M3, iOS 17 and macOS 14), so an MP4 stays
necessary, and VP9 already fits the size targets; Playwright's own video recording films the whole slide at 25 frames a second
and 1x, and its text was visibly softer.