Skip to content

JSONL / NDJSON viewer and LLM dataset validator

Inspect training corpora and log streams line by line, pinpoint syntax and message schema errors, and export clean train/validation splits locally.

How to use JSONL viewer & validator

  1. Paste records or load a corpus. Enter lines into JSONL / NDJSON lines, click Load sample JSONL, or drop a .jsonl or .ndjson file onto the drop zone.
  2. Pick a schema preset. Keep Auto-detect schema or select OpenAI Chat, Anthropic Messages, Prompt / Completion, or Generic NDJSON, then click Validate JSONL.
  3. Triage errors and inspect payloads. Filter rows with All rows, Errors / issues, or Valid only, click View to inspect parsed objects, or export CSV and Excel reports.
  4. Generate reproducible dataset splits. Set Train ratio (%) and Deterministic seed, toggle Keep schema-valid lines only if needed, click Preview split counts, and download ZIP or JSONL files.

Everything runs in this browser tab. Nothing you paste or open is uploaded or logged, so it's safe for keys, tokens and private data.

Why use JSONL or NDJSON instead of a giant JSON array?

Newline-delimited records parse independently line by line, isolating trailing commas or unclosed braces without aborting the entire dataset. Reading one object per row keeps multi-megabyte log streams and fine-tuning corpora manageable in browser memory while revealing line numbers immediately.

As defined in the JSON Lines on the Web specification, each non-empty line contains one self-contained UTF-8 object separated by a newline delimiter. Our JSONL and NDJSON viewer strips leading UTF-8 byte order marks (\uFEFF), normalizes CRLF and LF endings, and counts blank lines separately without misreporting trailing newlines as broken rows.

Traditional editors choke when one missing brace invalidates a 25 MB JSON array, hiding everything past that syntax fault. When isolated records need structural pretty-printing or syntax fixes before re-assembly, try browser JSON syntax repair without server uploads on individual objects, then paste them back into your corpus.

All line splitting, schema validation, BPE counting, and ZIP archiving execute locally inside a dedicated Web Worker (dev.worker.ts). Zero training prompts, telemetry logs, or file names cross the network during inspection.

Validating OpenAI, Anthropic, and generic JSONL schemas

Selecting Auto-detect schema under Dataset validation schema inspects the first 25 valid JSON objects to match conversational LLM training formats or fall back to generic NDJSON. You can override auto-detection at any time and trigger Validate JSONL across every line.

OpenAI Chat fine-tuning message rules

The OpenAI Chat validator enforces conversational rules documented in the OpenAI supervised fine-tuning guide. Every line must hold a non-empty messages array whose entries use recognized roles (system, developer, user, assistant, tool, or function).

Each message requires string or structured-content array data, except assistant turns carrying tool_calls or function_call objects. Every record also requires at least one assistant turn so training never ingests prompt-only conversations.

Anthropic Messages alternating turn checks

For Claude evaluation and tuning datasets, the Anthropic Messages preset validates turn ordering matching the Anthropic Messages API schema reference. An optional top-level system key accepts either a plain string or non-empty block array rather than embedding system instructions inside messages.

Conversations must start with a user turn, alternate strictly between user and assistant on every consecutive step, and end on an assistant reply. Misordered turns or duplicate consecutive roles trigger exact row-level errors immediately.

Prompt-completion pairs and generic NDJSON mode

Legacy completion corpora work under Prompt / Completion, requiring non-empty prompt and completion fields or instruction and output pairs. Choosing Generic NDJSON checks valid object or JSON value syntax per line for structured application logs and webhook events. If source records live in tabular sheets first, parse CSV spreadsheets into JSON records before running schema checks.

Validation preset Required root fields Role / turn constraint Blank / syntax behavior Primary use case
Auto-detect schema Inspects first 25 objects Selects OpenAI, Anthropic, or pair mode Skips blank lines cleanly Mixed unfamiliar datasets
OpenAI Chat Non-empty messages array Six valid roles; needs assistant Rejects missing content GPT fine-tuning runs
Anthropic Messages messages + optional system Strict user to assistant alternation Flags consecutive identical roles Claude conversation sets
Prompt / Completion prompt+completion or instruction+output Rejects blank input/output strings Flags syntax errors by line Legacy instruction corpora
Generic NDJSON Any valid JSON line value No chat role requirements Counts syntax errors only Log streams and telemetry

Token estimation and record-level error filtering

Validating syntax and schema structure runs alongside summary metrics in four live cards: Detected Schema, Valid JSON / Schema, Dataset Tokens, and Top Roles / Keys (tracking the 20 most frequent roles or top-level object keys).

Exact BPE counting vs fast multi-megabyte heuristics

To balance precision with speed in browser memory, records up to 1,500 lines run through exact o200k_base byte pair encoding inside the worker. Once a dataset exceeds 1,500 lines, the estimator switches to a calibrated character heuristic (3.8 characters per token) so 50 MB log dumps stay interactive under 500 ms.

When small system-prompt templates need granular token boundary color highlights or custom 128K context pricing comparisons, open them in our dedicated token counter. Combining corpus validation with exact prompt budgeting keeps API batch pipelines predictable.

Filtering malformed lines and exporting triage reports

Every line checked against the IETF RFC 8259 JSON specification feeds into Record inspector, which displays up to 250 filtered rows at a time and truncates raw preview strings at 140 characters. Toggle All rows, Errors / issues, or Valid only to isolate trailing commas or schema mismatches fast.

Clicking View opens a pretty-printed JSON modal with a Copy JSON button for instant repair. QA reviewers can also download jsonl-validation-report.csv (framed with a UTF-8 BOM and MITRE CWE-1236 formula neutralization) or jsonl-validation-report.xlsx (JSONL Report sheet) via Export CSV report and Export Excel report.

How does deterministic train and validation splitting work?

A seeded mulberry32 pseudorandom generator paired with Fisher-Yates shuffling assigns identical rows to training and holdout slices whenever your seed number stays fixed. Choosing a training percentage allocates exact partition counts before packing output files.

Adjust Train ratio (%) between 10% and 99% (80% default in the UI) and set Deterministic seed (42 default) before clicking Preview split counts to confirm row counts without writing files first. Unchecking Shuffle before split keeps chronological line order intact for time-series telemetry logs.

When ready, export standalone files with Download train.jsonl and Download val.jsonl, or click Download train + val ZIP to package dataset-train.jsonl and dataset-val.jsonl inside dataset-split.zip using level 6 DEFLATE compression. Checking Keep schema-valid lines only ensures broken records never contaminate fine-tuning jobs.

Tip: Enable Keep schema-valid lines only and record your Deterministic seed value in training run logs so teammates can reproduce exact holdout evaluation slices later.

Processing large NDJSON corpora offline in browser memory

Every stage from file drop to archive generation operates strictly on local bytes in browser RAM (Uint8Array slices inside dev.worker.ts), avoiding server uploads that leak proprietary customer payloads or PII telemetry. Because line splitting, token estimation, and JSON parsing execute off the main UI thread inside our JSONL viewer, even multi-megabyte webhook dumps keep page scrolling responsive.

Filtering previews inside Record inspector at 250 visible DOM rows prevents layout stalls even when inspecting tens of thousands of NDJSON records. Blank separator lines and trailing newlines are tallied accurately without mutating source text or flagging false syntax failures at end-of-file.

Teams preparing private evaluation logs or internal agent transcripts retain deterministic train/validation bundles straight in Downloads folders without external SaaS accounts. If downstream analytics pipelines need column-oriented storage instead of row-oriented NDJSON logs, inspect matching analytical warehouse outputs using our columnar Apache Parquet dataset inspector locally as well.

Questions

Why do JSONL and NDJSON files validate faster than huge JSON arrays?

Line-delimited JSON isolates every record onto its own newline instead of wrapping millions of rows inside one array bracket pair. When line 412 has an unclosed brace, the validator flags that exact row while continuing to inspect every remaining record cleanly.

How does auto-detect schema choose between OpenAI and Anthropic formats?

Auto-detect scans up to 25 valid JSON objects at the top of your file. Records carrying a top-level system field or alternating user and assistant messages classify as Anthropic Messages, while developer, tool, function, or tool_calls fields select OpenAI Chat.

When does the token counter switch from exact BPE to estimation?

Up to 1,500 parsed records, the worker computes exact o200k_base byte pair encoding totals across every line. Above 1,500 rows, the estimator switches to a fast character heuristic around 3.8 characters per token so multi-megabyte log streams stay responsive.

How does deterministic seed shuffling keep train and validation splits reproducible?

Enabling Shuffle before split drives a mulberry32 pseudorandom number generator initialized from your Deterministic seed integer before running a Fisher-Yates shuffle. Using the same seed on unchanged valid records always partitions the exact same lines into train and validation sets.

Can I filter out invalid training lines before downloading train.jsonl?

Yes. Checking Keep schema-valid lines only excludes every line flagged with a syntax parse failure or missing role requirement before shuffling and partitioning, ensuring dataset-train.jsonl and dataset-val.jsonl only contain clean records ready for training.

Why are empty lines skipped during line-by-line JSONL validation?

Log emitters and hand-edited JSONL datasets often leave trailing newlines or blank separator lines between batches. The parser strips UTF-8 byte order marks, ignores blank lines in the active record list, and reports blank line counts separately.

Updated