diff --git a/README.md b/README.md index 7bbc3d4..ec3afeb 100644 --- a/README.md +++ b/README.md @@ -1,22 +1,43 @@ # LAS Stream Viewer -Stream very large (10 GB+) LAS well-log files **line by line** in the browser. A Java 21 / -Quarkus backend indexes and streams the file with a constant, tiny memory footprint; a React + -Vite UI renders millions of lines via virtualization and plays them back as a live SSE stream. +A drilling-data viewer for **very large (10 GB+) LAS well-log files**. A Java 21 / Quarkus backend +indexes the file in a single streaming pass (constant, tiny memory footprint); a React + Vite UI +presents the data three ways: -Built for the Pason-style LAS 2.0 logs in `Desktop\LAS files` (up to ~12.5 GB, 426 curves, -~2.5 M rows). Nothing about the design assumes the file fits in memory. +- **๐Ÿ“ˆ Log Plot** โ€” a multi-track drilling strip-chart / log plot (the way an engineer reads a well). +- **โŠ• Crossplot** โ€” WOB-vs-ROP (and any X/Y) drilling-optimization scatter, colored by depth/time. +- **๐Œ Raw / QC** โ€” line-by-line virtualized view with live SSE streaming and whole-file search. + +Built for the Pason-style LAS 2.0 logs in `Desktop\LAS files` (up to ~12.5 GB, ~426 curves, +~2.5 M rows). Nothing in the design assumes the file fits in memory. ## Why it scales | Concern | Approach | |---|---| -| Open a 12.5 GB file | **Open-in-place** (no copy) โ€” or resumable chunked upload for remote files | -| Random access into the file | One-pass **sparse byte-offset index** (a checkpoint every 256 lines โ‰ˆ 80 KB for 2.5 M lines) | -| Indexing memory | Single streaming pass, 1 MiB buffer โ€” independent of file size | -| Streaming memory | One `BufferedReader` advanced sequentially; lines pushed over **SSE** with backpressure | -| Browser memory | Virtualized list (only visible rows in the DOM) + a capped (20 K-line) LRU cache | -| Reading a line range | Seek to nearest checkpoint, skip โ‰ค 255 lines โ€” effectively O(1) | +| Open a 12.5 GB file | **Open-in-place** (no copy) โ€” or resumable 16 MiB chunked upload | +| Random line access | One-pass **sparse byte-offset index** (a checkpoint every 256 lines โ‰ˆ 80 KB for 2.5 M lines) | +| Curve plots over millions of samples | **Pyramid** of min/max **and mean** built per 32-row bucket during the same pass | +| Honest decimation | **min/max per pixel, never averaging** โ€” a 2 s gas kick or torque spike survives when zoomed out | +| Indexing memory | Single streaming pass, 1 MiB buffer + tiny index + tens-of-MB pyramid โ€” independent of file size | +| Raw streaming | One `BufferedReader` advanced sequentially; lines pushed over **SSE** with backpressure | +| Browser memory | Virtualized line list + capped (20 K-line) cache; plots fetch only the visible window, decimated | + +## Drilling features + +- **Channel-role resolver** maps the ~426 raw Pason mnemonics to drilling roles (ROP, WOB, RPM, + torque, MSE, SPP, flow, total gas + C1โ€“C5, gamma, inclination/azimuth, stick-slip/vibration, pit + gain-loss, on-bottom, โ€ฆ) with sensible **physical default scales** so sensor-glitch spikes clip + instead of flattening the trace. +- **Log Plot**: index vertical (depth **or** time, with a toggle), value horizontal in side-by-side + tracks; min/max envelope + mid trace; **scrollbar** + wheel-zoom + Fit to navigate; **auto-fit** + (scale each track to the visible window); hover **crosshair + readout**; **curve picker** to + customize tracks; **EDR-style replay** (curves scroll like a rig strip-chart); **ฮฃ Stats** panel + (min/avg/max per channel over the window). +- **Crossplot**: bucket-mean X vs Y (default WOB vs ROP) colored by depth/time/channel, **on-bottom + only** filter, robust (1stโ€“99th pct) auto-ranges so outliers don't blow up the axes, colorbar, hover. +- **Robust axes/depth**: hole-depth is jump-capped so one garbage `DEPT` sample can't poison the + depth axis; an axis is only offered when its real extent is non-degenerate. ## Layout @@ -26,11 +47,15 @@ las-stream-viewer/ โ”œโ”€ run.ps1 dev: Vite (:5173) + Quarkus (:8090) โ”œโ”€ build.ps1 prod: bundle UI into the jar, serve on :8090 โ”œโ”€ src/main/java/com/oiusa/las/ -โ”‚ โ”œโ”€ model/ LasFile, Curve, HeaderSection -โ”‚ โ”œโ”€ index/ LineIndex (sparse offsets), LineReader (random access) -โ”‚ โ”œโ”€ service/ FileStore, IndexService (one-pass scan), LasHeaderParser, UploadService -โ”‚ โ””โ”€ web/ File / Lines / Stream(SSE) / Search(SSE) / Upload resources -โ””โ”€ frontend/ React + Vite + TS, @tanstack/react-virtual +โ”‚ โ”œโ”€ model/ LasFile, Curve, HeaderSection, ResolvedRole +โ”‚ โ”œโ”€ index/ LineIndex (sparse offsets), LineReader, Pyramid (min/max/mean overview), RowParser +โ”‚ โ”œโ”€ service/ FileStore, IndexService (one-pass index+pyramid), LasHeaderParser, +โ”‚ โ”‚ ChannelRoles (role mapping), CurveDataService (decimation/crossplot), UploadService +โ”‚ โ””โ”€ web/ File / Lines / Stream(SSE) / Search(SSE) / Upload / Curve (roles, curve-data, crossplot) +โ””โ”€ frontend/src/ + โ”œโ”€ components/ App, IngestPanel, FileList, Section, WellInfo, ChannelList, + โ”‚ HeaderPanel, Viewer (raw), LogPlot, Crossplot + โ””โ”€ api.ts, types.ts, las.ts, styles.css ``` ## Run (dev) @@ -40,18 +65,16 @@ cd C:\Users\Dell\Desktop\las-stream-viewer .\run.ps1 ``` -This installs UI deps on first run, starts Vite in a new window, and runs the Quarkus dev server. +Installs UI deps on first run, starts Vite in a new window, runs the Quarkus dev server. Open **http://localhost:5173**. -To run the pieces by hand: +By hand: ```powershell -# backend $env:JAVA_HOME="C:\Program Files\Java\jdk-21.0.11" $mvn = "C:\Users\Dell\.m2\wrapper\dists\apache-maven-3.9.9-bin\4nf9hui3q3djbarqar9g711ggc\apache-maven-3.9.9\bin\mvn.cmd" -& $mvn quarkus:dev -# frontend (separate window) -cd frontend; npm install; npm run dev +& $mvn quarkus:dev # backend :8090 +cd frontend; npm install; npm run dev # frontend :5173 (separate window) ``` ## Run (production, single port) @@ -65,22 +88,34 @@ java -jar target\quarkus-app\quarkus-run.jar ## Using it -1. **Open on disk** โ€” browse the server filesystem (constrained to `las.allowed-roots`, default your - home dir) and click a `.las` file. It opens in place; indexing starts immediately and the header - appears as soon as it's parsed. -2. **Upload** โ€” drag a file in; it's streamed to the server in 16 MiB chunks. -3. Watch the **LAS header** (version/well/curve metadata) populate in the sidebar. -4. In the viewer: **โ–ถ Stream** plays lines server-side over SSE (speed slider = lines/sec), scroll - freely through all lines (ranges fetched on demand), jump to **Top / Data / End** or any line, - and **Search** the whole file (streamed matches, click to jump). +1. **Open a file** (left panel): **Open on disk** browses the server filesystem (constrained to + `las.allowed-roots`) and opens a `.las` **in place, no copy** โ€” ideal for the multi-GB logs. + Or **Upload** to stream a file in 16 MiB chunks. Indexing + the curve overview build in the + background; the well info and channels appear in the sidebar as soon as the header is parsed. +2. **๐Ÿ“ˆ Log Plot** โ€” scroll the scrollbar (or wheel-zoom / Fit) through depth or time, hover for a + readout, **โ–ถ Replay** to animate, toggle **auto-fit** and **ฮฃ Stats**, add channels via **๏ผ‹ Curves**. +3. **โŠ• Crossplot** โ€” pick X/Y/Color, toggle **on-bottom only** for a clean founder-point view. +4. **๐Œ Raw / QC** โ€” stream the raw lines (speed slider), jump to Top/Data/End/any line, search the + whole file. Good for verifying export integrity (column alignment, NULLs, etc.). + +Collapse the left panel with the **โฎœ** button in the top bar for a bigger workspace. + +## Notes & caveats + +- The channelโ†’role mapping is auto-resolved from mnemonics; verify with the **๏ผ‹ Curves** picker and + swap channels if a mapping looks off for your file. +- Data quality is the data's own: some exports have broken channels (e.g. a stuck `DEPT`), in which + case that axis is suppressed and the **Raw / QC** tab is the place to confirm. Sensor-glitch spikes + are handled by fixed physical scales (Log Plot) and robust percentile ranges (Crossplot). +- First index of a 12.5 GB file takes a few minutes (it parses ~40 channels/row for the pyramid); + shown with a progress bar. Smaller logs index in seconds. ## Configuration (`src/main/resources/application.properties`) | Key | Default | Notes | |---|---|---| | `quarkus.http.port` | `8090` | API (and prod UI) port | -| `las.data-dir` | `${user.home}/.las-stream-viewer` | where uploads live | +| `las.data-dir` | `${user.home}/.las-stream-viewer` | where uploaded files live | | `las.allowed-roots` | `${user.home}` | local files may only be opened from under these roots | | `las.index-stride` | `256` | lines per index checkpoint (smaller = faster seeks, larger index) | | `las.upload-chunk-size` | `16777216` | upload chunk size hint (16 MiB) | -```