# SQLite Recovery Benchmark – harness, test data and results

DOI: [10.5281/zenodo.23928446](https://doi.org/10.5281/zenodo.23928446)

Reproducible benchmark of eight SQLite recovery tools on the SQLite Forensic Corpus
(Nemetz et al., 2018; anti-forensic categories by Schmitt, 2018) and three false-positive
scenarios after Lee et al. (2025). Companion data set to:

> D. Pawlaszczyk: *Recovering Deleted and Manipulated Records from SQLite Databases:
> A Reproducible Benchmark of SQLite Forensic Tools*. Preprint, ScienceOpen (2026).
> The preprint DOI is listed under "Related works" of this Zenodo record.
> OCRID: 0000-0001-7485-7478

## Contents

| Path | Content |
|---|---|
| `benchmark-nemetz/corpus/` | 94 databases of the SQLite Forensic Corpus (categories 0A–0E, 11–19) and the three Lee et al. scenarios (`0F-paper.db`, `0B-paper.db`, `10-paper.db` with `-wal`/`-shm`) |
| `benchmark-nemetz/ground_truth*.json` | answer keys (from the corpus XML; category 17 rebuilt from the SQL scripts) |
| `benchmark-nemetz/run_tools.sh` | runs all tools on all databases and writes the report |
| `benchmark-nemetz/tools/setup.sh` | installs the compared tools at pinned versions |
| `benchmark-nemetz/tools/adapters/` | one adapter per tool: runs it and converts its output to a common TSV format |
| `benchmark-nemetz/tools/limit.py` | per-run limits (180 s wall clock, 4 GB RSS of the process group) |
| `benchmark-nemetz/tools/shims/` | inert stand-ins for GUI / unbuildable imports (see its README) |
| `benchmark-nemetz/score_tools.py` | scoring (deleted records, live data, Lee scenarios, robustness) |
| `benchmark-nemetz/run.sh`, `score*.py`, `FQLiteDump.java` | FQLite-only run and the earlier per-category scorers |
| `benchmark-nemetz/out/tools/<tool>/` | **results**: normalised output per database (`<db>.tsv`), stderr (`<db>.log`), exit code and run time (`runs.tsv`) |
| `benchmark-nemetz/out/tools/report.md`, `report.csv` | aggregated results as reported in the paper (report text in German) |
| `SHA256SUMS` | checksums of all files |

### Files in this record

The directory tree above is split into three archives so that each can be previewed on Zenodo;
unpacking all three into the same directory restores it.

| File | Content |
|---|---|
| `harness-and-data.zip` | `benchmark-nemetz/` without `out/`: harness, adapters, corpus, ground truth |
| `results.zip` | `benchmark-nemetz/out/tools/`: normalised output (`*.tsv`), `runs.tsv` per tool, `report.md`, `report.csv` |
| `logs.zip` | `benchmark-nemetz/out/tools/`: stderr of every run (`*.log`) |
| `README.md`, `LICENSE`, `LICENSE-DATA.md`, `CITATION.cff`, `SHA256SUMS` | as above |

The normalised format, the scoring rules and the fairness notes are described in
`benchmark-nemetz/TOOLS.md`.

## Tools and versions

| Tool | Version |
|---|---|
| FQLite | commit `827eb41` of https://github.com/pawlaszczyk/fqlite (identical carving code in release 5.11) |
| sqlite4n6 (SecurityRonin) | 0.10.1 (crates.io) |
| DC3 SQLite Dissect | 1.0.0 (PyPI) |
| bring2lite | `e876bf2` |
| SQLite GUI Analyzer | `df3c8bd` |
| NFI xsqlite | `3e8c3dd` |
| sqlite_extract | `30e37ba` |
| SQLite `.recover` | sqlite3 CLI 3.50.4 |

The tools themselves are not included; `tools/setup.sh` fetches them.

## Reproducing the results

The harness expects to sit in the root of an FQLite checkout (it builds FQLite with Gradle):

```sh
git clone https://github.com/pawlaszczyk/fqlite.git && cd fqlite
git checkout 827eb41
cp -r /path/to/this/record/benchmark-nemetz .
benchmark-nemetz/tools/setup.sh        # Python >= 3.9, git, cargo (sqlite4n6), JDK 21-24
benchmark-nemetz/run_tools.sh          # all tools x all databases, then out/tools/report.md
TOOLS="fqlite sqlite3" ONLY="0C-*" benchmark-nemetz/run_tools.sh   # subsets
```

The results in this record were produced on the following system:

| Component | Version |
|---|---|
| Hardware | Apple MacBook Air, M4, 32 GB RAM |
| Operating system | macOS 27.0.1 |
| Java (FQLite build and run) | OpenJDK 25.0.1 2025-10-21 LTS (build 25.0.1+13-LTS, 64-bit Server VM, mixed mode, sharing) |
| Python (tool virtual environment) | 3.12.2 |
| sqlite3 CLI (`.recover`) | 3.50.4 |
| sqlite4n6 | 0.10.1 (Rust/cargo build) |

Run times depend on the machine; recovery results do not.

## Licences

* Benchmark code (scripts, adapters, `FQLiteDump.java`): Apache License 2.0 (`LICENSE`), like FQLite.
* Results, ground-truth files and the Lee et al. scenario databases: CC BY 4.0 (`LICENSE-DATA.md`).
* SQLite Forensic Corpus databases: dedicated to the public domain by their authors
  (Digital Corpora, CC0). Please cite Nemetz et al. (2018) and Schmitt (2018) when using them.
* `tools/shims/PyQt5` is taken from SecurityRonin/sqlite-forensic (see `tools/shims/README.md`).

## References

* S. Nemetz, S. Schmitt, F. Freiling: A standardized corpus for SQLite database forensics.
  Digital Investigation 24 (2018), S121–S130. https://digitalcorpora.org/corpora/sql/sqlite-forensic-corpus/
* S. Schmitt: Introducing Anti-Forensics to SQLite Corpora and Tool Testing. IMF 2018.
* Seonghyeon Lee, Sooyoung Park, Insoo Lee, Jongmoo Choi, A comprehensive analysis and evaluation of SQLite deleted Record recovery techniques: A survey, Forensic Science International: Digital Investigation,Volume 55, 2025, 302031, ISSN 2666-2817, https://doi.org/10.1016/j.fsidi.2025.302031.