added refactor and setup command plans (Setup first)

2026-07-10 03:18:56 -07:00 · 2026-01-20 11:55:12 -08:00
parent de4772b0e5
commit 8344c2b4ed
4 changed files with 482 additions and 0 deletions
@@ -0,0 +1 @@
+
@@ -0,0 +1,232 @@
+
+Based on the code you uploaded, the current architecture is a **"One-Shot Headless"** model. Every time a query runs, the CLI (or Daemon) spawns a new `analyzeHeadless` process, initializes the JVM, loads the project, runs a script, and shuts down.
+
+This is robust in terms of isolation (if it crashes, it doesn't affect the next run), but it is **slow** (high latency due to JVM startup) and **fragile** (parsing `stdout` mixed with Ghidra logs).
+
+Here is a roadmap to make the CLI significantly more robust and deeply integrated with the IDE.
+
+---
+
+### Part 1: Improving Robustness (The "Persistent Bridge" Architecture)
+
+To make this robust, you need to move from "Spawn Process -> Parse Stdout" to "Spawn Process -> Connect via Socket -> Keep Alive". This prevents the overhead of restarting Ghidra for every command.
+
+#### 1. Create a "Ghidra Bridge" Python Script
+
+Instead of many small scripts (`get_list_functions.py`, etc.), create one master Python script that runs an infinite loop inside Ghidra (Headless or GUI) and listens for JSON commands.
+
+**File:** `src/ghidra/scripts/bridge.py` (New file)
+
+```python
+# @category Bridge
+# @keybinding
+# @menupath Tools.Start CLI Bridge
+# @toolbar
+
+import socket
+import json
+import threading
+from ghidra.util.task import ConsoleTaskMonitor
+from ghidra.app.decompiler import DecompInterface
+
+# Define your command handlers here
+def handle_functions(args):
+    # ... (Logic from your existing get_list_functions_script)
+    return [{"name": "example", "addr": "0x1234"}]
+
+def handle_decompile(args):
+    # ... (Logic from your existing decompile script)
+    return {"code": "int main() { ... }"}
+
+COMMANDS = {
+    "functions": handle_functions,
+    "decompile": handle_decompile
+}
+
+def start_server(port=12345):
+    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+    s.bind(('127.0.0.1', port))
+    s.listen(1)
+    print(json.dumps({"status": "ready", "port": port}))
+
+    while True:
+        conn, addr = s.accept()
+        try:
+            # Read line-based JSON
+            f = conn.makefile()
+            while True:
+                line = f.readline()
+                if not line: break
+                
+                try:
+                    req = json.loads(line)
+                    cmd = req.get("command")
+                    args = req.get("args", {})
+                    
+                    if cmd in COMMANDS:
+                        result = COMMANDS[cmd](args)
+                        response = {"status": "success", "data": result}
+                    else:
+                        response = {"status": "error", "message": "Unknown command"}
+                        
+                    conn.sendall(json.dumps(response) + "\n")
+                except Exception as e:
+                    conn.sendall(json.dumps({"status": "error", "message": str(e)}) + "\n")
+        except:
+            pass
+        finally:
+            conn.close()
+
+if __name__ == "__main__":
+    # If running in GUI, run in background thread to not freeze UI
+    if isRunningHeadless():
+        start_server()
+    else:
+        t = threading.Thread(target=start_server)
+        t.start()
+
+```
+
+#### 2. Update `HeadlessExecutor` to manage a Lifecycle
+
+Modify `src/ghidra/headless.rs` or `src/daemon/process.rs`. Instead of just `Command::new()`, you need a struct that holds the `Child` process and a `TcpStream`.
+
+* **Startup:** Spawn `analyzeHeadless` with the `bridge.py` script.
+* **Handshake:** Wait for the specific JSON `{"status": "ready"}` on stdout.
+* **Execution:** Connect to the port via TCP. Send requests as JSON lines.
+* **Cleanup:** Kill the child process on daemon shutdown.
+
+This eliminates `stdout` parsing issues because data transfer happens over a clean TCP socket.
+
+---
+
+### Part 2: Better IDE Integration (Bi-directional Control)
+
+Currently, your CLI talks to a headless instance. The user wants to see results in the GUI.
+
+#### 1. Shared Project Locking
+
+Ghidra does not allow a Headless instance and a GUI instance to have write access to the same project simultaneously.
+
+* **Robustness Fix:** The CLI should detect if the GUI is open.
+* **Strategy:** If the GUI is open, the CLI should **not** spawn a headless instance. Instead, it should connect to the *Bridge* running inside the GUI.
+
+#### 2. Context Synchronization (CLI -> GUI)
+
+Add commands to the Bridge script that manipulate the GUI state.
+
+**Update `bridge.py`:**
+
+```python
+def handle_goto(args):
+    addr_str = args.get("address")
+    addr = currentProgram.getAddressFactory().getAddress(addr_str)
+    
+    # Check if we are in GUI mode
+    if not isRunningHeadless():
+        from ghidra.framework.plugintool import PluginTool
+        state = state # Ghidra injects 'state'
+        tool = state.getTool()
+        if tool:
+            tool.firePluginEvent(...) # Or simpler:
+            # This often requires the script to be run via the Ghidra Script Manager
+            currentLocation = ProgramLocation(currentProgram, addr)
+            tool.setGoTo(currentLocation)
+    return {"status": "moved"}
+
+def handle_highlight(args):
+    # Set background color of address range
+    if not isRunningHeadless():
+        setBackgroundColor(addr, Color.RED)
+
+```
+
+**Update Rust CLI (`src/cli.rs`):**
+Add a command `ghidra focus <address>` which sends the `goto` command to the bridge.
+
+#### 3. Automatic Discovery
+
+How does the CLI know if the GUI is running?
+
+1. **Port Scanning:** The Rust CLI can try to connect to the default Bridge port (e.g., 12345).
+2. **Logic:**
+* Try `TcpStream::connect("127.0.0.1:12345")`.
+* If successful -> **GUI Mode**. Send commands there.
+* If failed -> **Headless Mode**. Check if Daemon is running. If not, start Daemon (which spawns Headless Bridge).
+
+
+
+---
+
+### Part 3: Robustness Improvements in Rust
+
+#### 1. Fix Output Parsing (`src/ghidra/headless.rs`)
+
+Your current `extract_json_from_output` relies on counting braces. This is risky if the program being analyzed contains strings with braces.
+
+**Improved approach (if not using Bridge):**
+Wrap the output in a unique delimiter in the Python script.
+
+*Python Script:*
+
+```python
+print("---GHIDRA_CLI_START---")
+print(json.dumps(data))
+print("---GHIDRA_CLI_END---")
+
+```
+
+*Rust (`headless.rs`):*
+
+```rust
+fn extract_json_from_output(&self, output: &str) -> Result<String> {
+    let start_marker = "---GHIDRA_CLI_START---";
+    let end_marker = "---GHIDRA_CLI_END---";
+    
+    let start = output.find(start_marker)
+        .ok_or(GhidraError::ExecutionFailed("Missing start marker".into()))? 
+        + start_marker.len();
+        
+    let end = output.find(end_marker)
+        .ok_or(GhidraError::ExecutionFailed("Missing end marker".into()))?;
+        
+    Ok(output[start..end].trim().to_string())
+}
+
+```
+
+#### 2. Typed Responses with `serde`
+
+In `src/ghidra/data.rs`, strictly enforce optionals. If Ghidra scripts fail (e.g., decompilation error), they should return a standard error object.
+
+```rust
+#[derive(Deserialize)]
+#[serde(tag = "status")]
+enum BridgeResponse<T> {
+    #[serde(rename = "success")]
+    Success { data: T },
+    #[serde(rename = "error")]
+    Error { message: String },
+}
+
+```
+
+Update `src/daemon/queue.rs` to parse this wrapper before returning the inner string.
+
+---
+
+### Part 4: Implementation Plan
+
+Here is the recommended order of operations to upgrade your tool:
+
+1. **Implement the Bridge Script:** Create `scripts/ghidra_bridge.py`.
+2. **Update Daemon to support "Long-Running" Process:**
+* Modify `DaemonState` to hold a `Child` process handle.
+* Modify `HeadlessExecutor` to check if the bridge is up; if so, use TCP; if not, spawn it.
+
+
+3. **Add GUI Commands:** Add `goto`, `highlight`, and `select` to the bridge and `src/cli.rs`.
+4. **Integration Test:** Open Ghidra GUI, run the bridge script manually. Then run `ghidra query functions` from your terminal. It should return results instantly using the GUI's memory.
+
+This transforms your tool from a "Batch Processor" to a "Live Assistant."
@@ -0,0 +1 @@
+
@@ -0,0 +1,248 @@
+
+Here is the implementation plan to add a `ghidra setup` command that automates the downloading and installation of Ghidra.
+
+### 1. Update Dependencies
+
+First, we need to add crates for HTTP requests, file downloading, zip extraction, and progress bars.
+
+**Action:** Update `Cargo.toml`
+
+```toml
+[dependencies]
+# ... existing dependencies ...
+reqwest = { version = "0.11", features = ["json", "stream", "rustls-tls"] }
+zip = "0.6"
+futures-util = "0.3" # For handling download streams
+indicatif = "0.17"   # For progress bars
+
+```
+
+### 2. Update CLI Definition
+
+Add the `setup` command to the argument parser.
+
+**Action:** Modify `src/cli.rs`
+
+```rust
+// In enum Commands
+#[derive(Subcommand, Clone, Serialize, Deserialize, Debug)]
+pub enum Commands {
+    // ... existing commands ...
+    
+    /// Download and setup Ghidra automatically
+    Setup(SetupArgs),
+}
+
+// Define arguments
+#[derive(Args, Clone, Serialize, Deserialize, Debug)]
+pub struct SetupArgs {
+    /// Specific Ghidra version to install (e.g., "11.0"). Defaults to latest.
+    #[arg(long)]
+    pub version: Option<String>,
+
+    /// Installation directory. Defaults to standard data directory.
+    #[arg(long, short = 'd')]
+    pub dir: Option<String>,
+
+    /// Skip Java check
+    #[arg(long)]
+    pub force: bool,
+}
+
+```
+
+### 3. Create Setup Module
+
+Create a new module to handle the download and extraction logic. This keeps `main.rs` clean.
+
+**Action:** Create `src/ghidra/setup.rs`
+
+This file will contain logic to:
+
+1. **Check for Java**: Run `java -version` to ensure prerequisites are met.
+2. **Fetch Release Info**: Query the GitHub API (`https://api.github.com/repos/NationalSecurityAgency/ghidra/releases/latest`) to get the download URL.
+3. **Download**: Stream the zip file with a progress bar using `reqwest` and `indicatif`.
+4. **Extract**: Unzip the file to the target directory using `zip`.
+5. **Detect Installation**: Find the actual Ghidra folder inside the zip (usually `ghidra_X.X.X_PUBLIC`).
+
+**Sketch of `src/ghidra/setup.rs`:**
+
+```rust
+use std::path::{Path, PathBuf};
+use std::fs::File;
+use std::io::Write;
+use anyhow::{Context, Result, anyhow};
+use futures_util::StreamExt;
+use indicatif::{ProgressBar, ProgressStyle};
+
+pub async fn install_ghidra(version: Option<String>, target_dir: PathBuf) -> Result<PathBuf> {
+    // 1. Resolve Version & URL (GitHub API or hardcoded fallback for specific versions)
+    let (download_url, filename) = resolve_version_url(version).await?;
+    
+    // 2. Download File
+    let zip_path = target_dir.join(&filename);
+    download_file(&download_url, &zip_path).await?;
+    
+    // 3. Extract
+    let install_path = extract_zip(&zip_path, &target_dir)?;
+    
+    // 4. Cleanup zip
+    std::fs::remove_file(zip_path)?;
+    
+    Ok(install_path)
+}
+
+async fn download_file(url: &str, path: &Path) -> Result<()> {
+    let client = reqwest::Client::new();
+    let res = client.get(url).send().await?;
+    let total_size = res.content_length().unwrap_or(0);
+    
+    let pb = ProgressBar::new(total_size);
+    pb.set_style(ProgressStyle::default_bar()
+        .template("{msg}\n{spinner:.green} [{elapsed_precise}] [{wide_bar:.cyan/blue}] {bytes}/{total_bytes} ({eta})")?
+        .progress_chars("#>-"));
+    pb.set_message(format!("Downloading from {}", url));
+
+    let mut file = File::create(path)?;
+    let mut stream = res.bytes_stream();
+
+    while let Some(item) = stream.next().await {
+        let chunk = item?;
+        file.write_all(&chunk)?;
+        pb.inc(chunk.len() as u64);
+    }
+    pb.finish_with_message("Download complete");
+    Ok(())
+}
+
+fn extract_zip(zip_path: &Path, target_dir: &Path) -> Result<PathBuf> {
+    // Uses 'zip' crate to extract
+    // Returns the path to the extracted 'ghidra_X.X.X' directory
+}
+
+pub fn check_java_requirement() -> Result<()> {
+    // Exec "java -version" and check output
+}
+
+```
+
+### 4. Integrate Module
+
+Expose the new module.
+
+**Action:** Modify `src/ghidra/mod.rs`
+
+```rust
+pub mod setup;
+// ... existing code ...
+
+```
+
+### 5. Implement Handler in Main
+
+Connect the CLI command to the logic and update the configuration upon success.
+
+**Action:** Modify `src/main.rs`
+
+1. Add to the `run` match arm:
+```rust
+// In run() function
+Commands::Setup(args) => handle_setup(args).await,
+
+```
+
+
+2. Implement `handle_setup`:
+```rust
+async fn handle_setup(args: cli::SetupArgs) -> anyhow::Result<()> {
+    println!("Ghidra Setup Wizard");
+    println!("===================");
+
+    // 1. Check Java
+    if !args.force {
+        if let Err(e) = ghidra::setup::check_java_requirement() {
+            eprintln!("Warning: Java prerequisite check failed: {}", e);
+            eprintln!("Ghidra requires JDK 17+. Continue anyway? [y/N]");
+            // ... input confirmation logic ...
+        }
+    }
+
+    // 2. Determine Install Directory
+    let install_base = if let Some(d) = args.dir {
+        PathBuf::from(d)
+    } else {
+        // Default to XDG_DATA_HOME/ghidra-cli/ghidra
+        dirs::data_local_dir()
+            .ok_or(anyhow::anyhow!("Could not determine data directory"))?
+            .join("ghidra-cli")
+            .join("ghidra")
+    };
+
+    std::fs::create_dir_all(&install_base)?;
+
+    // 3. Install
+    println!("Installing to: {}", install_base.display());
+    let final_path = ghidra::setup::install_ghidra(args.version, install_base).await?;
+
+    // 4. Update Config
+    let mut config = Config::load()?;
+    config.ghidra_install_dir = Some(final_path.clone());
+    config.save()?;
+
+    println!("\nSuccess! Ghidra installed at: {}", final_path.display());
+    println!("Configuration updated.");
+
+    // 5. Verify
+    println!("\nVerifying installation...");
+    // Reuse existing doctor logic or verify_installation()
+    let client = GhidraClient::new(config)?;
+    client.verify_installation()?;
+    println!("Verification passed!");
+
+    Ok(())
+}
+
+```
+
+
+
+### 6. Make Main Async-Aware for Sync Commands
+
+Currently `run` is synchronous, but `reqwest` is async.
+
+* The `main` function is already `#[tokio::main]`.
+* We need to change `run(cli: Cli)` to `async fn run(cli: Cli)`.
+* Most existing handlers in `run` are synchronous; calling them from an async function is fine.
+* However, `handle_setup` needs to be awaited.
+
+**Refactoring:**
+Change the signature of `run` in `src/main.rs`:
+
+```rust
+async fn run(cli: Cli) -> anyhow::Result<()> {
+    match cli.command {
+        // ...
+        Commands::Setup(args) => handle_setup(args).await, // New async handler
+        _ => {
+            // Existing sync handlers can wrap in simple blocks if needed, 
+            // or just be called directly as they return Result
+            match cli.command {
+                Commands::Query(args) => handle_query(args),
+                // ... rest of sync commands
+                _ => Ok(()) 
+            }
+        }
+    }
+}
+
+```
+
+### Summary of Workflow
+
+1. User runs `ghidra setup`.
+2. CLI checks for Java.
+3. CLI fetches latest release URL from GitHub.
+4. CLI downloads ~300MB+ zip file showing progress.
+5. CLI unzips it.
+6. CLI updates `config.yaml` automatically setting `ghidra_install_dir`.
+7. User can immediately run `ghidra quick binary.exe`.