GG-UI — Design
Home | Edit Timeline | Threading | ECS & Storage | Rendering
GG-UI is the engine's retained-mode UI library: the foundation for the editor, in-game UI, and text-first tools. It replaced Dear ImGui and ImGuizmo in the editor.
Reference pages: Core · Rendering · Interaction · Styling and Widgets. Status: see RoadmapRoadmap.
Summary
GG-UI is built from the same mechanisms as the rest of Trinyx: the spreadsheet storage model, change-frequency tiers, published frames, dirty bits, GPU-driven predicate → scan → scatter, Buffer Device Address, and lifetime tiers. It behaves and reads like every other Trinyx subsystem.
- Zero-cost idle. Nothing is laid out, painted, or uploaded unless something changed.
- Cost follows the change, not the document. One changed label costs one node whatever else is on screen.
- World UI at constant CPU cost. Health bars and nameplates for thousands of entities are expanded on the GPU from slab data, and stay consistent with the displayed frame, including rollback and time travel.
- One renderer for UI and 2D. Screen UI, world UI, and 2D sprites share one primitive format.
- Multi-window and Wayland-first. Windows live on the main thread; docking is designed around Wayland's rules.
- Text is first-class. Editable text buffers, a code editor, consoles, and a cell-grid mode that also renders to a real terminal.
Module Boundary
| Layer | Location | Depends on | Contains |
|---|---|---|---|
| UI core | GGUI static library (src/GGUI) | A small foundation layer: math, containers, string IDs, chunked storage, Fixed32 | Tree and storage, mutation queue, layout, styling, interaction, accessibility rows, snapshots, primitive output. No World, Registry, renderer, or SDL |
| Text | GGUIText | FreeType, HarfBuzz, SheenBidi | Shaping, line breaking, the glyph atlas |
| Platform | src/Runtime/Platform | SDL3 | Windows, input, IME, DPI, clipboard, popups, file dialogs |
| Render backend | engine renderer | Vulkan | The UI pass, the texture table |
| Engine integration | src/Runtime/UI | UI core + engine | UIHost, the theme, the console, the inspector |
The core meets the rest through three interfaces: platform events in, UI primitives out, and scheduling. The GGUI target can't include engine headers; CMake enforces it. That keeps the core headless-testable and usable by standalone tools without a World.
Trinyx Principles, Applied to UI
| Engine principle | How GG-UI uses it |
|---|---|
| One data model, many consumers | One primitive format for screen UI, world UI, and sprites; one snapshot drives rendering, hit-testing, and accessibility |
| Spreadsheet storage | NodeIndex columns, SoA rows within chunks |
| Tiers by change frequency | Structure and layout inputs (cold) vs computed results (published) |
| Published frames between threads | The UI update publishes snapshots; Sentinel hit-tests and Encoder renders them |
| Dirty bits | Layout and publish queues; an idle UI touches nothing |
| GPU-driven compaction | World-UI template expansion and sprites use predicate → prefix_sum → scatter |
| BDA everywhere | Primitive buffers addressed by BDA; images by bindless index |
| OOP facade over a data-oriented substrate | Widget functions and typed widget structs over UI storage; authors never see the layout |
| Lifetime tiers | UI documents declare ConstructLifetime tiers; FlowManager tears them down with their domain |
| Determinism boundary | Presentation UI stays outside determinism and never writes sim state; sim-bound UI runs deterministically inside it |
Why Not ImGui
| Problem | Root cause |
|---|---|
| Detached windows vanish or can't be interacted with | ImGui's viewport windows were created on Encoder while SDL events are pumped on Sentinel. SDL3 requires window calls on the main thread, and on Win32 a window's messages go to the thread that created it |
| Floating windows don't snap | ImGui docking offers drop targets only |
| Layout reset is unreliable | DockBuilder rebuilds interact with imgui.ini restore |
| Immediate mode | UI state is rebuilt every frame on the render thread; no place for retained structure, bindings, or GPU-side expansion |
The window-thread problem isn't ImGui-specific: any multi-window UI must create and pump its windows on the main thread.
Constraints From the Engine
| Constraint | Consequence |
|---|---|
OS windows must be created and pumped on Sentinel. Wayland adds no client-side positioning, no global cursor position, and xdg_popup for menus | The platform layer lives on Sentinel. Docking and popups follow Wayland's rules |
| Sentinel polls at 1000 Hz and must never block | No layout or paint on Sentinel: only routing and hit-testing against published data |
| Sim input is a deterministic, timestamped stream | The UI decides what it consumes on Sentinel, before anything reaches InputBuffer |
| The sim is authoritative and rollback-capable | UI never writes sim state. It reads the published frame and acts through commands |
Slab memory is owned by a World and written only by Brain | UI storage isn't in the slab |
Architecture
Sentinel (1000Hz) UI update (display rate, jobs) Encoder ───────────────── ────────────────────────────── ─────── Platform windows (SDL3) Apply queued tree mutations Read latest snapshot Pump events Re-run what changed, bindings Upload changed ranges Route input: Layout UI pass per window hit-test last snapshot ───► Paint ───► (after the scene) UI takes it, or forward Publish snapshot to InputBuffer (sim)
- Tree mutations from any thread are queued and applied at the UI update: one pattern for all state mutation, like the spawn handshake.
- Window and platform requests go to Sentinel through a non-blocking queue; results come back as events.
- Hit-testing on Sentinel uses the last published snapshot, which is what the user sees.
Contexts and Lifetimes
A UIContext is an independent tree, storage, and snapshot chain, owned by the engine and not by a World: UI outlives World resets, spans PIE Worlds, and renders loading screens when no World exists.
UI documents declare a ConstructLifetime tier (Level / World / Session / Persistent). FlowManager destroys Level- and World-scoped UI with their domain; documents that survive a World reset get the same OnWorldTeardown / OnWorldInitialized callbacks to rebind. There is no separate UI lifetime vocabulary.
Editor Integration
The editor UI sits on the edit-mode contracts (Edit Timeline) and adds no second path into world state.
| Need | Mechanism |
|---|---|
| Read ECS state | The published read frame, never Brain's write frame |
| Change ECS state | FieldEdit batches and EntityLifetimeCommand, applied by Brain |
| Undo / redo | UndoHistory transactions |
| Interaction boundaries | The UI owns begin / commit / cancel for every interaction and opens and closes the transaction from it. Cancel restores the original |
| Live drags | The widget keeps its own value while dragging, so nothing reads back a stale frame |
| Time travel | Timeline UI drives the read-frame override; world UI follows because it is expanded from the displayed frame |
Deterministic UI
GG-UI is also the basis for 2D and UI-driven games (card, puzzle, strategy), where layout and hit-testing feed gameplay and must resimulate bit-identically.
| Presentation UI | Sim-bound UI | |
|---|---|---|
| Examples | Editor, menus, HUD chrome | Card hands and boards, UI that decides gameplay outcomes |
| Update rate | Display rate | The sim's fixed step, on Brain |
| Input | Platform events routed on Sentinel | The sim's input stream |
| Math | float | Fixed32 |
| Gameplay state | None; it acts through commands | In the slab (Temporal tier) |
Commitments that can't be retrofitted, and so hold now:
- Layout is written against a numeric type chosen per document. No hidden
floatin shared paths. - The core never reads a clock; time arrives through
AdvanceTime. - Evaluation order is by node, never by hash-container iteration.
- Text metrics enter sim-bound layout at one boundary, with fonts pinned per build.
Sim-bound documents on Brain are not built yet. The layout determinism test is.
Memory Model
UI storage is dedicated, built on the engine's storage principles, and not in the World slab. The slab was rejected:
| Issue | Detail |
|---|---|
| Lifetime | The slab belongs to a World; UI must outlive Worlds |
| Writer and rate | Brain is the only writer, at the fixed step. UI mutates at display rate from other threads |
| Rollback | Resim would sweep presentation-only data |
| Sizing | Slab capacity is split into SystemGroup arenas; UI would need a new arena in every config |
| Defrag | Moves EntityCacheIndex; UI handles would need rehydration |
Each UIContext owns chunks of 64 slots, allocated on demand and freed when empty, like archetype cold storage. No fixed capacity, no per-node heap allocation, and nodes never move.
- Handles are a stable index plus a generation. Freed slots are reused lowest-first to keep chunks dense.
- Rows are SoA within a chunk, grouped by how often they change.
- Sparse rows (
NodeSparse<T>) hold what few nodes have: a node without a value costs four bytes. - Per-node cost: 483 bytes including its share of four snapshots.
World UI and 2D
Not built. The primitive format and the binding model are fixed so these can be added without touching the core.
World UI is expanded on the GPU with the predicate → prefix_sum → scatter pattern:
- Template: a primitive list in local space plus bindings driven by entity fields through a fixed set of GPU-evaluable ops (Core).
- Entity side: a Volatile component (
CWorldWidget { TemplateID, AnchorOffset, Flags }). - Predicate: active + has widget + frustum + distance.
- Prefix sum over visible entities × primitive counts.
- Expand into the UI primitive buffer, using the interpolated positions scatter produced, or widgets jitter against meshes.
Consequences: constant CPU cost; always consistent with the displayed frame, including rollback, time travel, and undo. Arbitrary strings are shaped once on the CPU and placed by the GPU. Transient world UI (damage numbers) goes through TemporalEvents and the PresentationReconciler, so mispredicted numbers fade with an Anti-Event.
2D is the same path as a sprite renderer: sprite entities instanced from the slab, an orthographic camera, sort layers.
UI renders after scene reconstruction at output resolution. It must never pass through temporal reconstruction or upscaling, which is what smears text.
Text, Code, and Terminal
- Text buffers and editing are built (Interaction).
- Code editor (planned): incremental tokenizer, LSP client (clangd), diagnostics, folding, breakpoint gutter.
- Consoles are built (Interaction).
- Cell-grid mode and a terminal backend (planned): cell-grid documents rendered to ANSI/VT with no GPU, so dedicated servers and CI get a live console from the same documents. This is why the core emits renderer-agnostic primitives.
Code ↔ Node Graph
The node graph and C++ are two views of one program. The C++ source is the source of truth; the graph is a live projection of it.
- Code changes re-parse incrementally and update the graph, keeping each node's identity through its source anchor.
- Graph changes become minimal text edits applied as one transaction, so undo covers both views and comments and formatting survive.
- Today's paths are one-way and lossy. The round-trip model gets its own design document.
Platform Layer
On Sentinel, over SDL3, behind an engine-owned interface: no header above the platform layer includes SDL. SDL3 already handles the hard Wayland surface (fractional scale, xdg_popup, text-input-v3), and native backends would cost months for parity.
- Multiple windows, one swapchain each.
- UI units. A window's UI scale is its monitor's display scale × the user setting (
UIScale,--ui-scale). Layout and text re-rasterise at a new scale; nothing is bitmap-stretched. - Per-window input routing, text input and IME, clipboard (including the X11 / Wayland primary selection), system cursors, drop receive, popup windows, file dialogs, window ownership (keep-above). All asynchronous: a request from any thread, the answer as an event.
- Closing a window closes its popups first, so the renderer releases every surface before the OS window goes.
Native code behind the same interface:
| Gap | Platform | Status |
|---|---|---|
| Custom title bar with Snap Layouts | Win32 | Done. The app publishes its maximize button's rect (SetMaximizeRegion) and it hit-tests as the OS button |
| Modal move / resize loop stalls Sentinel | Win32 | Done. The platform ticks Sentinel about 60 times a second during the loop (SetModalLoopTick). The OS still waits up to ~0.5 s on a title-bar press before entering it |
| Starting a drag between windows | Wayland | Planned: a wl_data_device source of our own (Docking Under WaylandDocking Under Wayland) |
| IME candidate window placement | All | SetTextInputArea from the focused widget's caret; unverified per IME |
Linux Verification
Built and checked on Windows. Everything goes through SDL and is expected to work, but X11 and Wayland runs are incomplete. Use the platform test window (--platform-test); its controls are listed in PlatformProbe.h. The routing logic (PlatformRouter) has no OS dependency and is covered by headless tests on every platform.
| Area | Known platform differences |
|---|---|
| Windows | Wayland: the compositor places top-level windows; SetPosition does nothing |
| Custom chrome | Wayland: no Snap Layouts equivalent. Standard windows need client-side decorations (libdecor) on GNOME |
| Popups | Wayland: xdg_popup; the compositor may reposition or dismiss one, and a menu must open in response to input |
| Keyboard focus | Wayland: raising a window needs an activation token and may be refused |
| Scale | Wayland: fractional scale, so logical and physical pointer positions differ. X11: one global scale |
| Clipboard | The primary selection exists only on X11 and Wayland |
Docking Under Wayland
| Capability | Win32 / X11 | Wayland |
|---|---|---|
| Dock inside a window | Yes | Yes |
| Tear off into a new window | Yes | Yes (the compositor places it) |
| Layout save / restore | Yes | Yes |
| Drag a tab between windows | Yes | Not yet: needs a drag source SDL doesn't expose (planned below) |
Capabilities are detected at run time.
Between windows, a tab drag finds the window under the pointer from window positions (UIHost::DockDocumentAt). Wayland gives neither window positions nor a global pointer, and during the press the source window keeps every pointer event. Only the compositor knows which surface is under the pointer, and it says so only during a wl_data_device drag. SDL 3.5 receives those drags but can't start one.
The plan, from reading SDL's Wayland backend; to be built and checked on Linux:
- Source. The platform layer binds its own
wl_data_device_managerandwl_seaton SDL'swl_display(SDL_PROP_GLOBAL_VIDEO_WAYLAND_WL_DISPLAY_POINTER). They share SDL's default queue, so SDL's event pump dispatches them. Awl_pointerof its own on that seat receives the same button events, which carry the serialstart_dragneeds. SDL keeps that serial private. - Start. When a tab leaves its window, start a drag from that window's
wl_surfacewith a source offering a private text MIME type and a token naming the drag. Optionally show a drag icon of the tab. - Target. SDL accepts text drops, so its own data device turns the drag into
SDL_EVENT_DROP_BEGIN,DROP_POSITION(window-local),DROP_TEXTandDROP_COMPLETEon whichever of our windows is under the pointer. These already reach the platform router; a drop carrying our token is routed toCarryOver/CarryEndinstead of a text drop. - Outside. The source's
cancelledevent means nothing took it: float the panel. The compositor places it. - Problems. Starting the drag sends
wl_pointer.leaveto the source window, and SDL answers that with a synthetic button release. The platform layer must swallow it, or the drag ends where it started. Another application may accept the text type;sendwrites nothing, so it receives an empty drop. Linkswayland-clientdirectly; SDL loads the same library at run time.
Roadmap
| # | Milestone | Status |
|---|---|---|
| M1 | Platform layer: multi-window, input routing, text input, DPI | Done on Windows; Linux verification open |
| M2 | UI core: storage, tree, mutation, layout, snapshots, accessibility rows, determinism test | Done |
| M3 | Rendering: primitive pass, clipping, bindless images, text | Done |
| M4 | Interaction: routing, focus, scrolling, text editing, popups, string tables, console, inspector | Done |
| M5 | Styling: widget definitions, tokens, rules, transitions, sheets with hot reload, the engine theme | Done |
| M5b–c | Core at scale: incremental publish and paint, documents on jobs, late documents leaving the frame | Done |
| M6 | Editor widgets: virtualised lists, trees, tables, tiles, property grid, panels, drag and drop, input actions | Done |
| M7 | Docking and multi-window | Done on Windows; step 6 to do on Linux |
| M8 | Editor migration, panel by panel; engine-rendered gizmo; engine FieldSource and string table | Done |
| M8b | Node graph canvas; code editor | Graph done (Core); code editor open |
| M9 | Game UI: game state in code, widget assets, the HUD elements, rich text and action prompts, style scopes, animation, starter widgets, the designer, configurable input, GPU world-UI templates and screen markers, sprites, sim-bound documents (Widgets) | Designed |
| M10 | Remove ImGui and ImGuizmo | Done |
| M11 | Accessibility bridge (UI Automation, AT-SPI) | |
| M12 | Visual logic in widget assets | |
| M13 | Terminal backend | |
| M14 | Code ↔ graph two-way sync |
M7: Docking and Windows
| Step | What | Status |
|---|---|---|
| 1 | The dock tree in one window: splits, tab groups, tabs dragged to rearrange, panels closed and reopened | Built (Core) |
| 2 | Named layouts, the default always available, saved as text | Built |
| 3 | Panels in their own OS windows: tear off, dock back, drag between windows, keep-above | Built |
| 4 | Dialogs, a message with choices, notices, the platform file dialog | Built (Core) |
| 5 | Menu-bar hover and arrow switching; a toolbar overflow menu; more tabs than fit | Built (Core) |
| 6 | Dragging a tab between windows on Wayland | To do on Linux (planplan) |
Untested by hand: the real file dialog, tab dragging with a real mouse, and layout restore across a restart. Nothing after the first Linux pass has been built on Linux.
M8: The Editor on GG-UI
The shell moved first, so there was one dock from the start (EditorShell); panels not yet moved over were drawn by ImGui into areas of it until M10. The editor waits for every document each frame (an update budget of 0) and its frame on Encoder reads what was just published; the shell's handlers post their work to that frame.
Shortcuts are rebindable actions under Editor.*, with defaults most editors share:
| Action | Default |
|---|---|
| Open, save, save as, save all | Ctrl+O, Ctrl+S, Ctrl+Shift+S, Ctrl+Alt+S |
| Undo, redo | Ctrl+Z, Ctrl+Y (or Ctrl+Shift+Z) |
| Duplicate, delete | Ctrl+D, Delete |
| Select (no gizmo), move, rotate, scale; cycle move / rotate / scale | Q, W, E, R; Space |
| World or local space | X |
| Frame the selection; drop it to the floor | F; End |
| Maximize the panel under the pointer | Shift+Space |
| Play (or go on playing); step a paused session | F5; , and . |
| Stop playing; give the pointer and keys back to the editor while playing | Escape; Shift+F1 |
| Fly the camera (held with the right button) | W, A, S, D, E, Q |
While playing, the game has the keyboard, except chords of actions kept from the game (UIHost::): Stop and Release Pointer. Their commands exist only while playing, so Escape keeps its usual meaning otherwise. Keys that also fly the camera (W, E, Q) don't switch tools while it flies.
Open: hiding entities and Constructs in the scene, so a designer can place things that a trigger reveals. It needs its own design: whether a hidden entity collides, ticks, replicates, and can be picked in the editor.
| Step | What | Status |
|---|---|---|
| 1 | The shell: title bar with menus, workspaces and play controls, the dock, editor actions (Editor.*) in Keybinds.ini | Built |
| 2 | EntityFields (a FieldSource over an entity's reflected components: reads the published frame, edits as FieldEdit, one undo step per interaction); TnxStringTable (a file per language, reloaded on change) and an interface language setting | Built (Core, Interaction) |
| 3 | The scene viewport as a ui.Viewport (picking and camera fly through its pointer events); ui.Canvas (lines, polylines, arcs, triangles) and the gizmo drawn on it over the viewport, replacing ImGuizmo; the grid drawn in the scene with depth, as a ground plane to judge size by | Built (Rendering, editor) |
| 4 | Panels: Log, Engine Stats, Outliner (a tree showing parenting), Details, Content Browser, Debugger | Built (editor); parenting waits for attachments in the engine |
| 5 | Dialogs and overlays: open / save, import, unsaved changes, asset issues, prefab save, the command palette, the frame budget strip | Built (editor); file choices use the platform dialog, the budget strip became the status bar |
| 6 | Play in editor: the local game in the viewport; Authority and Owner views as dock panels | Built (editor) |
| 7 | Construct, Entity and Prefab editors; the component generator | Built (editor); their node graphs moved over in M8b |
String files are hand-editable, one per language, so anyone can add a language:
# content/strings/en.strings Editor.Outliner.Title = Outliner Editor.Scene.Open.Tip = Open a .tnxscene file
Not Placed
- A first frame in proportion: build, layout, and publish of a large document from scratch cost about 0.7 microseconds a node, and need the three phases to walk chunks rather than nodes.
- Charts.
- A full engine input action (
TnxInputAction); see Interaction.
Performance
Measured on one core, optimised build, fixed-metric text. GGUI_Scale holds every run to these; Performance Targets has the full list.
| For 100,000 nodes | |
|---|---|
| An update with nothing changed | 0 |
| One changed label: build, layout | 0.0024 ms, 0.0019 ms |
| One changed label: publish | 0.015 ms |
| Hit-test | 0.008 ms |
| Finding a node in a snapshot | under 0.0001 ms |
| Build from scratch | 26.5 ms |
| A list scrolled slightly, 200 nodes showing | 0.015 ms, no rows written |
| Paint | |
|---|---|
| One changed label with 40,000 primitives showing | 0.0017 ms |
| 10,000 primitives from scratch | 0.20 ms |
| Widgets | |
|---|---|
| A list of a million rows | Built in 0.28 ms; 0.003 ms per frame scrolled within a row |
| A table of a million rows, six columns | Built in 0.83 ms; 0.011 ms per frame scrolled within a row |
| A late document (250 ms update) | The other documents' frames took at most 0.05 ms each |
| Text shaping, eight threads | 1,800 strings in 9 ms (16 ms on one thread) |
How It Got There
Design choices that produced those numbers, and the problems that forced them:
- Snapshots share unchanged row chunks. Publishing one changed label used to write every node (9.7 ms at 100,000 nodes).
- Rows hold unscrolled positions; scroll offsets are stored separately. Scrolling a box used to rewrite the row of everything inside it (20 ms for a 100,000-node list).
- Nodes keep their primitive slots; draw order is a separate list. A label that grew by one glyph used to move every primitive drawn after it.
- Each document caches its own resolved styles (
GGLooks), so a style table is only read during updates and documents sharing one update in parallel. - The host never touches a document that is updating, and never changes what one reads. A slow document holds one frame, the one that discovers it, and none after.
- The text engine is three things, not one lock: a run table read by many, shaping with per-thread FreeType and HarfBuzz objects, and an atlas locked only to place a new glyph.
Problems found along the way, outside the UI:
- Job system: a quarter of the workers take physics jobs only, and a submitted job's wake went to any sleeping worker, so a job could wake one that couldn't take it and sit until something else was submitted. A coordinator that helps with its own jobs hid this. Physics-only workers now wait on their own signal.
- SheenBidi: its lock-free scratch-buffer stack handed one buffer to two threads under contention and crashed four runs in ten once shaping was parallel. It is built without that cache.
- Store forwarding: primitives written a field at a time and copied straight after stalled the copy; see Rendering.
Decision Log
| Decision | Choice | Why |
|---|---|---|
| Library | Custom, engine-native | Built from Trinyx's own mechanisms; one foundation for editor and game |
| Module boundary | A core library with no engine dependency; platform, renderer, and integration as engine layers | Headless tests, standalone tools, replaceable layers |
| Memory | Dedicated chunked storage per UIContext | Lifetime, writer and rate, rollback, sizing |
| Lifetimes | ConstructLifetime tiers | One lifetime vocabulary across the engine |
| World UI instance data | In the entity slab, expanded on the GPU | Constant CPU cost, consistent with the displayed frame |
| Frame position | After scene reconstruction, at output resolution | Text must never pass through TAA or upscaling |
| Threading | Sentinel routes; the UI update runs on jobs and publishes; Encoder renders | Sentinel never blocks; windows stay on the main thread |
| Text | FreeType + HarfBuzz + BiDi from the start | Shaping can't be retrofitted cheaply |
| Authoring | A C++ builder; widget assets as data built at runtime through it | Editor needs came first; assets give hot reload and the designer one format that already runs |
| Game data in UI | Assets declare typed contexts and read their fields from the published frame; game code supplies the contexts | No per-frame binding code; right across rollback and time travel |
| Windowing | SDL3 behind an engine-owned interface, with native code for known gaps | Wayland support; replaceable per subsystem |
| Editor reads and writes | Read the published frame; write through FieldEdit inside undo transactions | Brain stays the only slab writer; undo is one mechanism |
| Docking | One host-owned dock space; floating panels are always OS windows | The platform's snapping applies, and it is the only kind Wayland lets be placed |
| Notices and file dialogs | Services of the host, reached through handles | A window isn't a world's; a handler has no ui |
| Strings | String IDs with a default table, or source text as the key | IDs survive rewording; plain text needs nothing declared |
| Determinism | Two document modes: presentation and sim-bound | UI-driven games need bit-identical resimulation; presentation UI shouldn't pay for it |
| Code ↔ graph | C++ is the source of truth; the graph is a projection | Two-way editing without losing hand-written code |
Open Questions
- The op set for world-UI template bindings: what is the minimum useful set?
- The accessibility bridge: build on an existing library (AccessKit) or implement the platform APIs directly.
- How UI code touches engine state safely from a document updating on a job.