add workflow
This commit is contained in:
Benjamin Palko
parent
b8f96fc411
commit
ad57d055f2
2 changed files with 222 additions and 3 deletions
22
AGENTS.md
22
AGENTS.md
|
|
@ -62,19 +62,35 @@ other licence-restricted vendor files.
|
|||
|
||||
## Analysis workflow
|
||||
|
||||
The full reproducible procedure, with commands, is in
|
||||
[`docs/WORKFLOW.md`](docs/WORKFLOW.md). Follow it for every binary. In summary:
|
||||
|
||||
1. Route the target first: native executables/libraries go through REA
|
||||
`open_binary` (never pass a Ghidra project file as the target — pass the
|
||||
original binary path under `binaries/`).
|
||||
2. Work summary-first: `binary_overview` / summary detail before drilling into
|
||||
`analyze_function` or pseudocode. Reuse retained Evidence IDs instead of
|
||||
re-running identical analysis.
|
||||
3. Export raw decompilation artifacts (pseudocode, listings) into `decomp/`
|
||||
3. **Third-party libraries: every time one is discovered** (an imported DLL, a
|
||||
version string, RTTI class prefix, library error strings, or a PDB path),
|
||||
do the following before continuing analysis:
|
||||
- pin its version;
|
||||
- **search online for its source or headers** (official releases, git tags,
|
||||
vendor repos, clean-room equivalents);
|
||||
- download it into `third-party/`, or into the gitignored
|
||||
`third-party/_local/` if its licence forbids redistribution;
|
||||
- record it in `third-party/README.md`.
|
||||
|
||||
If no source exists, record that and the acquisition routes. See
|
||||
`docs/WORKFLOW.md` §3.
|
||||
4. Export raw decompilation artifacts (pseudocode, listings) into `decomp/`
|
||||
using per-binary subdirectories, e.g. `decomp/GameServer_Release_USA_x64/`.
|
||||
Keep filenames stable and function-address-based so results are reproducible.
|
||||
4. Track targets, artifact SHA-256 identities, and per-binary status in
|
||||
For full coverage, use the headless scripts in `decomp/scripts/`.
|
||||
5. Track targets, artifact SHA-256 identities, and per-binary status in
|
||||
`decomp/INVENTORY.md`; keep a per-binary finding ledger in
|
||||
`decomp/<name>/evidence.md`.
|
||||
5. Close the REA session (`close_binary`) when the investigation is complete.
|
||||
6. Close the REA session (`close_binary`) when the investigation is complete.
|
||||
|
||||
## Evidence and claims
|
||||
|
||||
|
|
|
|||
203
docs/WORKFLOW.md
Normal file
203
docs/WORKFLOW.md
Normal file
|
|
@ -0,0 +1,203 @@
|
|||
# Workflow — reverse-engineering a server binary
|
||||
|
||||
The repeatable process used for `Agent_Release_x64.exe`. Follow it for each
|
||||
binary in `decomp/INVENTORY.md`. Ground rules, layout and tooling are in
|
||||
`AGENTS.md`. What has been learned so far is in `docs/ARCHITECTURE.md`.
|
||||
|
||||
Every finding is recorded as **observed**, **inference** or **unknown**, with an
|
||||
Evidence ID or decomp file reference. Static analysis never establishes runtime
|
||||
behaviour.
|
||||
|
||||
---
|
||||
|
||||
## 0. Prerequisites (once per machine)
|
||||
|
||||
- Ghidra is at `/opt/ghidra`, and Java 21 and Node come from `mise.toml`.
|
||||
- REA MCP is registered project-locally in `opencode.json`.
|
||||
- Don't reinstall any of these; repair the configuration instead.
|
||||
- **Ghidra headless fails with `UnsupportedClassVersionError` (class file
|
||||
71.0):** its compiled-script cache was built by a newer JDK.
|
||||
- Move `~/.config/ghidra/ghidra_<ver>_DEV/osgi/compiled-bundles` aside.
|
||||
- Ghidra rebuilds the cache on the next run.
|
||||
|
||||
## 1. Identify the binary (cheap, before any decompilation)
|
||||
|
||||
```sh
|
||||
sha256sum binaries/<X>.exe
|
||||
objdump -p binaries/<X>.exe | grep -E 'DLL Name|Subsystem|MajorLinker|pdb'
|
||||
```
|
||||
|
||||
In REA:
|
||||
1. Run `open_binary` with the original path under `binaries/`.
|
||||
2. Run `binary_overview`.
|
||||
3. Run `search_strings` (regex) for config paths, log text, protocol names and
|
||||
class names.
|
||||
|
||||
Record the SHA-256, linker version, PDB path, imports and procedure count in
|
||||
`decomp/<X>/evidence.md`.
|
||||
|
||||
## 2. Determine language and toolchain
|
||||
|
||||
Check for these C++ markers:
|
||||
- RTTI descriptors (`strings -a | grep '^\.?AV'`)
|
||||
- `__CxxFrameHandler3`
|
||||
- an `MSVCP*.dll` import
|
||||
|
||||
All 16 servers are MSVC 2010 C++ (see `docs/ARCHITECTURE.md`). Re-check each
|
||||
new binary and record the result.
|
||||
|
||||
## 3. Discover third-party libraries and acquire their sources (**mandatory**)
|
||||
|
||||
Any library you can match against upstream source saves you from
|
||||
decompiling it, and gives names, struct layouts and intent that the binary
|
||||
lacks. **Every time a library is found, search for its source online before
|
||||
continuing.**
|
||||
|
||||
### 3a. Find candidates
|
||||
|
||||
| Signal | How |
|
||||
|---|---|
|
||||
| Imported DLLs | `objdump -p` DLL names, cross-checked against `binaries/*.dll` |
|
||||
| DLL version resources | `strings -el <dll> \| grep -A1 -E '^(FileVersion\|ProductName\|CompanyName)$'` |
|
||||
| Embedded version text | `strings -a \| grep -iE '<lib>/[0-9]\|part of OpenSSL\|libcurl/'` |
|
||||
| Statically linked C++ libraries | RTTI class prefixes (`TiXml*`, `Proud::*`, `ATL::*`) |
|
||||
| Library-specific strings | assert/error messages (`Proud::CNetServerImpl::…`, TinyXML error table) |
|
||||
| PDB paths | e.g. `…\OpenLibrary\openssl-1.0.0d\…`, which names the exact version |
|
||||
|
||||
### 3b. Pin the version
|
||||
|
||||
Use, in this order of preference:
|
||||
1. An explicit version string (VERSIONINFO, `"OpenSSL 1.0.0d 8 Feb 2011"`).
|
||||
2. A version in the PDB path.
|
||||
3. **Differential evidence:** download several candidate releases and compare a
|
||||
distinctive artefact with the binary. Examples are error-string tables,
|
||||
exported symbol lists and header constants.
|
||||
- Example: the TinyXML error table contains `Memory allocation failed.`.
|
||||
That entry exists in 2.5.x and was removed in 2.6, which gives 2.5.1–2.5.3.
|
||||
4. If none of these work, choose the closest era-compatible release and mark it
|
||||
**closest, not exact**.
|
||||
|
||||
### 3c. Search for the source
|
||||
|
||||
Search in this order:
|
||||
1. Official release archives (project site or SourceForge; check `old/` and
|
||||
`archeology/` paths).
|
||||
2. Official git tags (`git ls-remote --tags <repo>`), or a GitHub tag archive
|
||||
if the release tarball no longer exists.
|
||||
3. Vendor package repositories. These often contain headers only.
|
||||
4. Clean-room equivalents (for example mingw-w64 headers for Microsoft APIs)
|
||||
or open reimplementations (for example Wine).
|
||||
5. `gh search repos <lib>` for forks or mirrors. Treat these as unverified and
|
||||
label them that way.
|
||||
|
||||
Verify each download against an official checksum where one exists. Always
|
||||
record its SHA-256.
|
||||
|
||||
### 3d. Check the licence and place the files
|
||||
|
||||
| Licence | Location | Committed? |
|
||||
|---|---|---|
|
||||
| Open source (OpenSSL, curl/MIT, zlib, BSD, public domain) | `third-party/<lib>-<version>/` | Yes |
|
||||
| Proprietary or EULA-restricted (e.g. ProudNet) | `third-party/_local/<lib>-<version>/` | **No.** It is gitignored. Ask the user before fetching. |
|
||||
| No public source (XTrap, Apex, HASP runtime) | nothing downloaded | Record acquisition routes only |
|
||||
|
||||
Other rules:
|
||||
- Keep upstream files unmodified. If you must change one (for example curl's
|
||||
version stamp), record exactly what changed.
|
||||
- Read the licence before committing anything. If redistribution is
|
||||
prohibited, it goes in `_local/` or isn't fetched.
|
||||
|
||||
### 3e. Document it
|
||||
|
||||
Add a row or section to `third-party/README.md` with:
|
||||
- the observed identity in the binary, with where it was seen;
|
||||
- the version and how it was pinned;
|
||||
- the source URL, tag or commit, and SHA-256;
|
||||
- match quality (exact / closest / headers only);
|
||||
- the licence;
|
||||
- any local edits.
|
||||
|
||||
Also:
|
||||
- Mention the library in `docs/ARCHITECTURE.md` if it is statically linked.
|
||||
- Add it to the list in `AGENTS.md`.
|
||||
- If no source exists, add it to the "No public source" and "Acquisition
|
||||
routes" tables instead.
|
||||
|
||||
### 3f. Use it
|
||||
|
||||
- Label library functions in the decompiled output from known exports, class
|
||||
method tables, error strings and the matched headers.
|
||||
- Exclude labelled library code from application analysis.
|
||||
|
||||
## 4. Decompile
|
||||
|
||||
In REA, choose seeds from distinctive strings or imports and use
|
||||
`trace_feature` to find the functions that reference them. Then run
|
||||
`procedure_pseudo_code` / `batch_decompile` on those functions.
|
||||
|
||||
For full coverage, run Ghidra headless with the repo scripts:
|
||||
|
||||
```sh
|
||||
analyzeHeadless /tmp/opencode/ghproj X -import binaries/<X>.exe -overwrite \
|
||||
-scriptPath decomp/scripts -postScript ExportDecomp.java "$PWD/decomp/<X>/full" -deleteProject
|
||||
analyzeHeadless /tmp/opencode/ghproj X -import binaries/<X>.exe -overwrite \
|
||||
-scriptPath decomp/scripts -postScript VTableMap.java "$PWD/decomp/<X>/full/vtables.tsv" -deleteProject
|
||||
```
|
||||
|
||||
This produces:
|
||||
- `full/functions/<addr>.c` and `full/index.tsv` from `ExportDecomp.java`;
|
||||
- `full/vtables.tsv` from `VTableMap.java`, which maps class vftable slots to
|
||||
functions.
|
||||
|
||||
Generate `cpp/classes.hpp` from `vtables.tsv`, then check that it compiles:
|
||||
`g++ -fsyntax-only`.
|
||||
|
||||
## 5. Record evidence
|
||||
|
||||
In `decomp/<X>/`:
|
||||
1. Run `export_evidence_bundle` to write `evidence-bundle.json`.
|
||||
2. Extract the REA pseudocode to `<addr>.pseudo.c` and the overview to
|
||||
`binary_overview.json`.
|
||||
3. Write `evidence.md`: the findings table (conclusion → type → evidence), the
|
||||
proposed function names, and the open questions.
|
||||
4. Update the status and role of the binary in `decomp/INVENTORY.md`.
|
||||
|
||||
## 6. Lift to readable C++
|
||||
|
||||
`decomp/<X>/cpp/*.cpp` holds hand-lifted code for the analysed functions:
|
||||
- Cite the original address on each function.
|
||||
- Mark every name as inferred.
|
||||
- Record the original bugs you notice (leaks, races).
|
||||
|
||||
## 7. Port to Linux
|
||||
|
||||
`src/<name>/` builds the lifted code with Meson (see `src/README.md`). Keep a
|
||||
`platform.{hpp,cpp}` layer that maps each Win32 call to its nearest POSIX
|
||||
equivalent.
|
||||
|
||||
The per-binary `README.md` must contain:
|
||||
- a substitution table: original address → Win32 API → Linux replacement →
|
||||
behaviour difference;
|
||||
- what is stubbed or not ported, and why.
|
||||
|
||||
Build with `ninja -C build` and smoke-test it. A passing test proves only that
|
||||
the port works, not how the original behaves.
|
||||
|
||||
## 8. Close
|
||||
|
||||
Run `close_binary` when the investigation is complete. Before closing, export
|
||||
the Evidence bundle, because session annotations are temporary.
|
||||
|
||||
---
|
||||
|
||||
## Checklist per binary
|
||||
|
||||
- [ ] SHA-256, imports, linker and PDB recorded
|
||||
- [ ] Language confirmed
|
||||
- [ ] **Every third-party library identified, its version pinned, its source
|
||||
searched for and downloaded where licensing allows, and it documented in
|
||||
`third-party/README.md`**
|
||||
- [ ] Key functions decompiled; `full/` export done
|
||||
- [ ] `evidence.md`, `INVENTORY.md`, `docs/ARCHITECTURE.md` updated
|
||||
- [ ] Lift and Linux port updated, if in scope
|
||||
- [ ] Evidence bundle exported; REA session closed
|
||||
Loading…
Add table
Add a link
Reference in a new issue