Initial commit: LAS Stream Viewer (Quarkus backend + React log-plot UI)
This commit is contained in:
86
README.md
Normal file
86
README.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# LAS Stream Viewer
|
||||
|
||||
Stream very large (10 GB+) LAS well-log files **line by line** in the browser. A Java 21 /
|
||||
Quarkus backend indexes and streams the file with a constant, tiny memory footprint; a React +
|
||||
Vite UI renders millions of lines via virtualization and plays them back as a live SSE stream.
|
||||
|
||||
Built for the Pason-style LAS 2.0 logs in `Desktop\LAS files` (up to ~12.5 GB, 426 curves,
|
||||
~2.5 M rows). Nothing about the design assumes the file fits in memory.
|
||||
|
||||
## Why it scales
|
||||
|
||||
| Concern | Approach |
|
||||
|---|---|
|
||||
| Open a 12.5 GB file | **Open-in-place** (no copy) — or resumable chunked upload for remote files |
|
||||
| Random access into the file | One-pass **sparse byte-offset index** (a checkpoint every 256 lines ≈ 80 KB for 2.5 M lines) |
|
||||
| Indexing memory | Single streaming pass, 1 MiB buffer — independent of file size |
|
||||
| Streaming memory | One `BufferedReader` advanced sequentially; lines pushed over **SSE** with backpressure |
|
||||
| Browser memory | Virtualized list (only visible rows in the DOM) + a capped (20 K-line) LRU cache |
|
||||
| Reading a line range | Seek to nearest checkpoint, skip ≤ 255 lines — effectively O(1) |
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
las-stream-viewer/
|
||||
├─ pom.xml Quarkus 3.34.3, Java 21
|
||||
├─ run.ps1 dev: Vite (:5173) + Quarkus (:8090)
|
||||
├─ build.ps1 prod: bundle UI into the jar, serve on :8090
|
||||
├─ src/main/java/com/oiusa/las/
|
||||
│ ├─ model/ LasFile, Curve, HeaderSection
|
||||
│ ├─ index/ LineIndex (sparse offsets), LineReader (random access)
|
||||
│ ├─ service/ FileStore, IndexService (one-pass scan), LasHeaderParser, UploadService
|
||||
│ └─ web/ File / Lines / Stream(SSE) / Search(SSE) / Upload resources
|
||||
└─ frontend/ React + Vite + TS, @tanstack/react-virtual
|
||||
```
|
||||
|
||||
## Run (dev)
|
||||
|
||||
```powershell
|
||||
cd C:\Users\Dell\Desktop\las-stream-viewer
|
||||
.\run.ps1
|
||||
```
|
||||
|
||||
This installs UI deps on first run, starts Vite in a new window, and runs the Quarkus dev server.
|
||||
Open **http://localhost:5173**.
|
||||
|
||||
To run the pieces by hand:
|
||||
|
||||
```powershell
|
||||
# backend
|
||||
$env:JAVA_HOME="C:\Program Files\Java\jdk-21.0.11"
|
||||
$mvn = "C:\Users\Dell\.m2\wrapper\dists\apache-maven-3.9.9-bin\4nf9hui3q3djbarqar9g711ggc\apache-maven-3.9.9\bin\mvn.cmd"
|
||||
& $mvn quarkus:dev
|
||||
# frontend (separate window)
|
||||
cd frontend; npm install; npm run dev
|
||||
```
|
||||
|
||||
## Run (production, single port)
|
||||
|
||||
```powershell
|
||||
.\build.ps1
|
||||
$env:JAVA_HOME="C:\Program Files\Java\jdk-21.0.11"
|
||||
java -jar target\quarkus-app\quarkus-run.jar
|
||||
# open http://localhost:8090
|
||||
```
|
||||
|
||||
## Using it
|
||||
|
||||
1. **Open on disk** — browse the server filesystem (constrained to `las.allowed-roots`, default your
|
||||
home dir) and click a `.las` file. It opens in place; indexing starts immediately and the header
|
||||
appears as soon as it's parsed.
|
||||
2. **Upload** — drag a file in; it's streamed to the server in 16 MiB chunks.
|
||||
3. Watch the **LAS header** (version/well/curve metadata) populate in the sidebar.
|
||||
4. In the viewer: **▶ Stream** plays lines server-side over SSE (speed slider = lines/sec), scroll
|
||||
freely through all lines (ranges fetched on demand), jump to **Top / Data / End** or any line,
|
||||
and **Search** the whole file (streamed matches, click to jump).
|
||||
|
||||
## Configuration (`src/main/resources/application.properties`)
|
||||
|
||||
| Key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `quarkus.http.port` | `8090` | API (and prod UI) port |
|
||||
| `las.data-dir` | `${user.home}/.las-stream-viewer` | where uploads live |
|
||||
| `las.allowed-roots` | `${user.home}` | local files may only be opened from under these roots |
|
||||
| `las.index-stride` | `256` | lines per index checkpoint (smaller = faster seeks, larger index) |
|
||||
| `las.upload-chunk-size` | `16777216` | upload chunk size hint (16 MiB) |
|
||||
```
|
||||
Reference in New Issue
Block a user