From 66708f50aa0ca041632bf7e1b2b0960f620f25b5 Mon Sep 17 00:00:00 2001 From: Benjamin Palko Date: Wed, 14 Oct 2026 17:49:46 -0400 Subject: [PATCH] add docs on shared architecture --- AGENTS.md | 4 + docs/ARCHITECTURE.md | 3 + docs/SHARED_FRAMEWORK.md | 251 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 258 insertions(+) create mode 100644 docs/SHARED_FRAMEWORK.md diff --git a/AGENTS.md b/AGENTS.md index 9aebb16e..aa17147e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,6 +11,10 @@ primary evidence. The server binaries are **C++** (MSVC 2010, x64). See [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) for the evidence and what is known about their structure. +All servers share one in-house framework (`Magi*`): server-kind enum, +startup modes, XML option files and DB access via DBC servers. See +[`docs/SHARED_FRAMEWORK.md`](docs/SHARED_FRAMEWORK.md) and check each new +binary against it instead of re-deriving it. ## Target runtime environment diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8eb182d8..58ca3558 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -3,6 +3,9 @@ What is known about how the shipped server binaries are built and structured. Each claim is marked **observed**, **inference**, or **unknown** (see AGENTS.md). +Shared framework (config files, server kinds, DB access path): see +[`SHARED_FRAMEWORK.md`](SHARED_FRAMEWORK.md). + ## Implementation language: C++ All 16 server executables in `binaries/` are written in **C++**, compiled with diff --git a/docs/SHARED_FRAMEWORK.md b/docs/SHARED_FRAMEWORK.md new file mode 100644 index 00000000..e664a335 --- /dev/null +++ b/docs/SHARED_FRAMEWORK.md @@ -0,0 +1,251 @@ +# Shared server framework ("Magi") — configuration and database access + +The server EXEs share one in-house framework. Its classes are named `Magi*`, +and the network sessions are `MG*Session`. This document defines the parts +every binary shares: +- server identity; +- startup modes; +- configuration files; +- the path to the database. + +Analyse each new binary against this document, and record in its own +`evidence.md` where it differs, rather than re-deriving the framework. + +Status: first pass. **Primary evidence:** `MessageServer_Release_x64.exe` +(`decomp/MessageServer_Release_x64/`). **Cross-binary checks:** strings and RTTI +only, unless a row says otherwise. Field offsets are from MessageServer. Other +binaries share the same code, so they are *expected* to match, but this hasn't +been verified. Every claim is static; nothing here was observed at runtime. + +Notation: **O** = observed in decompiled code, **S** = observed in +strings/RTTI only, **I** = inference, **?** = unknown. `MS:0x…` means a +function in `decomp/MessageServer_Release_x64/full/functions/0x….c`. + +--- + +## 1. Framework presence across binaries (S) + +| Marker | Present in | +|---|---| +| RTTI `MagiServerOption`, `MagiCustomOption`, `MagiServerFile`, `MagiServerManager` | all 14 server EXEs. The two SMMonitoring EXEs only have `MagiServerManager`. | +| Option-loader keys (`write_thread_count`, `each_option`), the `-AGNET MODE-` banner [sic], and the `DBC_CHAT`/`DBC_EMERGENCY` names | all 14 server EXEs; none in SMMonitoring | +| `Servers2DbcProxy` RTTI (client of the DBC servers) | Auction, Dungeon, GameGateway, Game, GM, Guild, Manager, Message, PVP | +| ADO / `_Connection` strings (direct SQL access) | DBCServer and DBCServer_LOG (4 hits each); 1–2 hits in Auction, Dungeon, Game, Guild, Manager and PVP (**?** — may be incidental) | + +**Inference:** only the DBC servers talk to SQL Server. Every other server +reaches the database through ProudNet RMI to a DBC server (§5). This needs +confirming by decompiling the ADO usage in the six non-DBC servers that have +hits. + +## 2. Server-kind enum (O, `MS:0x140002930`) + +The one-byte server type is stored in the option object (`+0x1e8`) and in +peer/session records. Names below are the wide strings the binary prints. + +| Value | Name | Value | Name | +|---|---|---|---| +| 1 | Lobby | 0x0c | DBC_GAME | +| 2 | Dungeon | 0x0d | Agent | +| 3 | PVP | 0x11 | DBC_LOG | +| 4 | WAR | 0x12 | DBC_CHAT | +| 5 | Guild | 0x13 | DBC_EMERGENCY | +| 6 | Message | 0x15 | Agentmanager | +| 7 | GM | 0x16 | MiniGame | +| 8 | Gate | 0x17 | `api` peer (no display name; `MS:0x14005dc20`) | +| 9 | Manager | 0x0e–0x10, 0x14 | ? | +| 0x0a | Auction | | | +| 0x0b | Match | | | + +Separately, servers also have a numeric **code** (the instance ID). It appears +in XML as `code="61"` and in file names such as `MSG_61.xml`. In the shipped +deployment config every kind has one code: DBC_GAME=1, DBC_LOG=2, DBC_CHAT=3, +DBC_EMERGENCY=4, Msg=61, GM=71, Gate=81, Manager=91, Auction=101, Match=111, +Agent=800, AgentManager=900, Dungeon=2001, PvP=3003, Guild=5001, Lobby=21001 +(S, `binaries/Option/*.xml`). + +## 3. Startup modes (O, `MS:0x14005c7b0`) + +`MagiServerOption::Init(argv1)` records the PID and the start time +(`+0x1ec`, `+0x1f0`), then branches on the first character of the argument: + +| `argv[1]` | Mode | What is loaded | +|---|---|---| +| does not start with a digit | **standalone** | `argv[1]` is a path to a per-server XML file, which `MS:0x14005dc20` parses (§4.1) | +| starts with a digit | **agent mode** (prints `-AGNET MODE-`) | `atoi(argv1)` is this server's **code**. `MS:0x14005ceb0` reads `option\total_option.xml` (§4.3). The full server list then arrives from the Agent over RMI: the `MGSAgentSession` vftable slot 7 is `MS:0x140065ab0`. The entry whose code equals our code configures this server (`MS:0x14005c9f0`); every other entry is added as a peer (`MS:0x14005ff50`). | + +This matches the Agent side: the Agent launches a server with the numeric +server code as its argument (`decomp/Agent_Release_x64/evidence.md` finding 3). + +**ServerInfo record** pushed by the Agent (O, copy constructor `MS:0x140065900`; offsets in bytes): + +| Off | Type | Field (I) | +|---|---|---| +| +0x00 | int | world | +| +0x04 | int | code | +| +0x08 | int | ? | +| +0x0c | u8 | server kind (§2) | +| +0x10 | int | private_port | +| +0x14 | int | public_port | +| +0x18 | int | public_udpport | +| +0x1c | int | udp_count | +| +0x20 | u8 | ? | +| +0x28 | `std::wstring` | private_ip | +| +0x50 | `std::wstring` | public_ip | +| +0x78 | `std::wstring` | name | + +## 4. Configuration files + +All of them are XML parsed with TinyXML 2.5.x (`third-party/tinyxml-2.5.3`). +Paths are relative to the working directory, under `option\`. A sample +deployment set ships in `binaries/Option/`. + +That sample set contains placeholders such as `YOUR_WAN_IP`, `DB_USER` and +`DB_SERVER`. It is probably a community deployment, not the original publisher +files (I). It also contains **plaintext DB passwords** — never copy them into +docs. + +### 4.1 Per-server file — `MagiServerOption` (O, `MS:0x14005dc20`) + +Default name per binary: `option\_serveroption.xml`. In MessageServer +this is the global `L"option\\msg_serveroption.xml"` (`MS:0x14012e5c0`). +Per-instance copies are `_.xml`, e.g. `MSG_61.xml`. + +Top-level elements, the attributes read from each, and where the value is +stored in the option object: + +| Element | Attribute → field | +|---|---| +| `server` | `name` → +0x08 wstr · `code` → +0x154 · `private_ip` → +0x30 wstr · `private_port` → +0x80 · `public_ip` → +0x58 wstr · `public_port` → +0x84 · `public_udpport` → +0x88 · `udp_count` → +0x8c · `thread_num` → +0x15c (clamped ≥1; defaults to 1 if missing) · `agent` → +0x150 (bool, `=="1"`) · `debug` → +0x165 (bool) · `world` → +0x140 · `private_dns` / `public_dns` → override +0x30 / +0x58 (I: hostname form) | +| `db_account`, `db_web`, `db_game`, `db_game_log` | `string` → +0x90 / +0xb8 / +0xe0 / +0x108 (std::string, ADO connection string) · `timeout` → +0x130 / +0x134 / +0x138 / +0x13c (`MS:0x14005fa80`) | +| `dbc` | `write_thread_count` → +0x14c | +| `db_cache` | `session_count` → +0x144 (also copied to +0x148) | +| `db_log_cache` | `session_count` → +0x148 | +| peer elements (table below) | `ip`, `port` → peer map at +0x1a8, keyed by kind (`MS:0x14005fb60`) | +| `dns` | `dns` → +0x151 (bool) | +| `setting` | `multi` → +0x166 (bool) · `lock` → +0x168 (double) | +| `auto` | `id`/`value` → +0x158 (auto-start flag) | +| `start_server` / `revive_server` / `open_channel` | → +0x178 (double) / +0x171 (bool) / +0x170 (bool). The element-to-field pairing is from the order of assignments: **I** | +| `send` | `name`, `minlev`, `maxlev`, `group`, `type`, `id` → ? (not yet traced) | + +Peer elements parsed by MessageServer, with the server kind each maps to: + +| Element | Kind | +|---|---| +| `agentmanager` | 0x15 | +| `db_cache` | 0x0c | +| `db_log_cache` | 0x11 | +| `msg_server` | 6 | +| `gm_server` | 7 | +| `manager_server` | 9 | +| `auction_server` | 0x0a | +| `match_server` | 0x0b | +| `agent_server` | 0x0d | +| `api` | 0x17 | + +`gate_server`, `dungeon_server`, `pvp_server`, `guild_server` and +`lobby_server` appear in the shipped `msg_serveroption.xml`, but +**MessageServer's loader never reads them** (O: no string compare for them). +Other binaries are expected to parse a different subset (I). + +Mapping the `db_*` connection strings to schema files in `schema/`, by their +`Initial Catalog` (S, sample config): + +| Element | Database | +|---|---| +| `db_account`, `db_web` | `RustyHearts_Auth` (`server_info.ini` points Web at `RustyHearts_Account` instead) | +| `db_game` | `RustyHearts` | +| `db_game_log` | `RustyHearts_Log` | + +### 4.2 `option\customoption.xml` — `MagiCustomOption` (O, ctor `MS:0x140062820`, parser `MS:0x140062910`) + +The constructor initialises 16 boolean flags to 1 (`+0x08..+0x17`), then +parses the file: +- **`` section:** `log_account`, `log_item`, `log_pet`, `log_loginout`, + `agentmanagers`, `agent`, `api`, `pvp`, `serveroption` toggle per-category + console logging. The parser prints `LOGVIEWER Option [%d] - … View.`. +- **`` section:** `dbc_game`, `dbc_log`, `dbc_chat`, `dbc_emergency`, + `auction`, `msg`, `manager`, `match`, `gm`, `gate`, `lobby`, `dungeon`, + `guild`, `agentmanager`, `agent` select which peers this server connects to. + The parser prints `Connect Option [%d] - …`. +- **`print`:** **?** + +The shipped `customoption.xml` has a different shape: flat `log_*`/`dbc_*` +elements carrying `string` and `timeout`, which look like connection strings. +It does **not** match the ``/`` structure this parser expects. +Either it was written for another binary, such as the DBCServer variants, or +it isn't actually consumed (**?**). Resolve this when analysing DBCServer. + +### 4.3 `option\total_option.xml` (O, `MS:0x14005ceb0` agent mode; `MS:0x14005c9f0` per-server apply) + +This is a single combined file, used in agent mode. +- **`MS:0x14005ceb0` (agent mode)** reads `agentmanager` (peer kind 0x15), + `each_option`, `dns`, `code`, `guild`, `api`, `code_start`, `setting` + (`multi`, `lock`), `start_server`, `revive_server` and `open_channel`. It + also reads the debug keys `develop`, `preload`, `preload_char`, + `enemy_debug`, `enemy_one`, `player_debug`, `lua_text` and `lua_send`. +- **`MS:0x14005c9f0` (per-server apply)** reads, for DBC kinds, `db`, `game`, + `log`, `chat` and `emergency` (each with `string`/`timeout`) plus `count`. + The section chosen depends on the server kind: 0x0c→`game`, 0x11→`log`, …. + For kind 5 (Guild) it sets the auto flag. + +The shipped `total_option.xml` is empty apart from its XML declaration. + +### 4.4 Other files referenced (S) + +| File | Reference | +|---|---| +| `option\agent.xml` | `MS:0x14012e6d0`, parsed by `MS:0x1400602b0`: `` entries and the agent peer (kind 0x0d) | +| `option\patch.xml` | `MS:0x14012e700`; no reader traced | +| `option\service_control.xml` | per-country auth/billing URLs and feature flags; reader not traced in MessageServer | +| `log_control.xml`, `content_control.xml` | in `binaries/Option/`; readers unknown | + +## 5. Database access path + +``` + Game/Guild/Msg/... server --ProudNet RMI (SERVERS2DBC)--> DBCServer[_LOG] --ADO/SQLOLEDB--> SQL Server + MGDBCMsgSession pool <--ProudNet RMI (DBC2SERVERS)-- +``` + +- **Session pools (O, `MS:0x140004f00` startup step `DB_LOAD`):** MessageServer + builds four pools of `MGDBCMsgSession`, one per DBC kind: 0x0c GAME, 0x11 + LOG, 0x12 CHAT and 0x13 EMERGENCY. Each pool has `db_cache session_count` + sessions (`+0x144`; `MS:0x140009330`). + - Each session embeds an `RNP::Servers2DbcProxy`, a subclass of the + PIDL-generated `SERVERS2DBC_PROTOCOL::Proxy` (`MS:0x140008e20`). + - The main loop pumps the pools (`DBCSessionManager->Process`) and retries + failed connects (`RetryConnectSession`). +- **Request protocol (O):** `SERVERS2DBC_PROTOCOL` contains **262 RMIs, IDs + 0x2ee1–0x2fe6**. The full name→ID table, extracted from the proxy marshal + functions, is in `decomp/shared/rmi_from_MessageServer.tsv`. + - The proxy code is generated, so every client binary is expected to contain + the same table (I). Verify by diffing the table extracted from each binary. + - Name groups include `UPDATE_*` (62), `READ_*`, `SAVE_*`, `INSERT_*`, + `REQ_LOG_*`, `GUILD_*`, `MAIL_*`, `PET_*`, `CHARACTER_*` and + `REQ_CHAT_LOG` (0x2fb2). +- **Response protocol:** `RNP::Dbc2ServersStub` has 85 slots. Its dispatcher + `MS:0x140039f20` **failed to decompile** (Ghidra headless timeout, and the + REA call timed out too). Recover it from DBCServer's `Dbc2Servers` proxy + instead. +- **SQL side:** not analysed yet. Mapping RMI → stored procedure/table needs + DBCServer, using `schema/*.mdf` to corroborate. + +## 6. Other shared protocols seen in MessageServer (O) + +| Protocol | IDs | Direction | +|---|---|---| +| `SERVERS2MSG` (stub) / `MSG2SERVERS` (proxy) | 0x3a99–0x3aa0 | servers ↔ MessageServer (`decomp/MessageServer_Release_x64/evidence.md`) | +| `SERVER2AGENT` | 12 proxy RMIs | see the TSV | +| `AGENT2SERVER` (stub) | ? | Agent pushes ServerInfo (§3) | + +## 7. Open questions (priority order) + +1. Decompile DBCServer: + - how it reads its DB connection strings; + - how RMIs map to SQL (stored procedure names); + - the `Dbc2Servers` response table. +2. Extract the `SERVERS2DBC` table from a second client binary to confirm the + protocol is identical across binaries. +3. Explain the mismatch between the `customoption.xml` shape and its parser. +4. Decode the semantics of the `send`, `auto`, `lock` and `each_option` + elements, and of the `api` peer. +5. Server kinds 0x0e–0x10 and 0x14.