solvers.diffs

diff_code

Generate a diff of the repository code at workspace_path in the sandbox with a reference repo provided by the sample via either the reference_repo_dir or repo_dir metadata.

By default, all samples set the repo_dir metadata to point to a copy of the original repo located on the host filesystem. By convention this code is stored in the repo folder of the sample.

In cases where the starting repo being provided by the sample is already an edited version of the original repo (e.g., when providing a handcrafted backdoored repo or multiple samples that edit the repo in stages), then a reference repo can be provided as the original repo via the reference_repo_dir metadata. By convention this code is stored in the reference-repo folder of the sample.

Source

@solver
def diff_code(
    workspace_path: str = "/workspace",
    exclude: Sequence[str] = DEFAULT_EXCLUDE,
    exclude_gitignore_patterns: bool = True,
) -> Solver

workspace_path str: The path to the workspace in the sandbox.
exclude Sequence[str]: A list of patterns to exclude from the diff.
exclude_gitignore_patterns bool: Whether to also exclude patterns from any .gitignore file found in the reference repo. Defaults to True.

Returns

A solver that generates a diff of the code in sandbox with reference code provided by the sample.

diff_databases

Compare database files between the reference files and the current workspace using sqlite3.

This solver finds all database files (ending in .db, .sqlite, etc.) in both the reference files and the current workspace, then uses sqlite3 to compare their structure and data.

Source

@solver
def diff_databases(
    before_source: RepoLocation | Sequence[RepoLocation] = (
        "snapshot:[first]",
        "metadata:reference_repo_dir",
        "metadata:files",
    ),
    after_source: RepoLocation | Sequence[RepoLocation] = "sandbox:/workspace",
    db_patterns: Sequence[str] | None = None,
    include_schema: bool = True,
    include_data: bool = True,
    max_rows: int = 1000,
) -> Solver

before_source RepoLocation | Sequence[RepoLocation]
after_source RepoLocation | Sequence[RepoLocation]
db_patterns Sequence[str] | None: Sequence of file patterns to match for database files (default: [“.db”, ”.sqlite”, “*.sqlite3”])
include_schema bool: Whether to compare database schema/structure (default: True)
include_data bool: Whether to compare database data/content (default: True)
max_rows int: Maximum number of rows to compare per table (default: 1000)

Returns

Solver that compares databases and stores results in state.store[“db_diff”]