Skip to content

Developer

How to bring up a dev/test loop for TileFoundry, and how we work on it.

1. Env

Use a dedicated tilefoundry-dev conda env (Python ≥ 3.12 + torch built against your local CUDA). Cloning an existing torch+CUDA env is easiest. Do not install tilefoundry into base.

conda create --clone <torch_env> -n tilefoundry-dev
conda activate tilefoundry-dev

Record where your env lives in a gitignored .dev-env at repo root (informational; nothing reads it automatically):

echo CONDA_ENV=tilefoundry-dev > .dev-env

2. Install

git submodule update --init --recursive   # third_party/cutlass
python -m pip install -e '.[test]'

pyproject.toml declares runtime deps (apache-tvm-ffi, torch), the [test] extra (pytest), and the jinja *.j2 templates as package data. After this import tilefoundry resolves to src/tilefoundry/... and plain pytest tests/ works from repo root. Do not set PYTHONPATH — in particular PYTHONPATH=tests:src shadows stdlib types via tests/types/__init__.py.

3. Tests

pytest tests/ runs the entire suite — unit, codegen, nvcc-compiled, and GPU end-to-end — with no marker gating. There is no --gpu flag or -m nvcc opt-in: a machine without nvcc or a CUDA device fails (not skips) the compile / e2e tests, so end-to-end coverage is never silently hidden. Per-test dump output lands under test_results/{file_stem}/{test_name}/... (gitignored).

Common inner loops:

pytest tests/passes tests/ir tests/ir_types -q   # IR / passes edits
pytest tests/codegen tests/runtime -q            # codegen-text edits
pytest tests/e2e -q                              # runtime header / host wrapper edits

4. Development workflow

  • main is the integration branch. Develop on task/<id>-<short> branches and merge via PR.
  • Discuss requirements / design / trade-offs before any code change.
  • Plan: draft under docs/plans/<name>.md against docs/plans/TEMPLATE.md, then run scripts/finalize_plan_context.py <plan> to inject the plan-level Preflight + per-milestone policy ACs from docs/policies/project-policy.json.
  • Reviewers read the finalized plan as the implementation contract.
  • After approval, implement milestone-by-milestone: claim a task, cut the branch, code + test, post results, open the PR.
  • One commit per milestone batch; commit messages carry the milestone tag. No --amend, no force-push to main, no --no-verify.
  • Claiming done requires test evidence.

5. Principles

Cross-cutting rules every change MUST honour. Each section is a short bullet list — keep it that way.

Code comments

  • Documentation lives in a module / class / function docstring. A # line is for a tool (ruff: / noqa / type: / pragma: / mypy: / fmt: / isort:) or a shebang, never for a reader. In C / C++ / CUDA the docstring's place is a Doxygen block above the declaration (/** */, ///, trailing ///<); // and /* */ are not used.
  • A docstring spends at most 8 prose lines and stays inside 100 columns. Google sections (Args: / Returns: / Raises: / Yields: / Attributes:) are structure, not prose, and do not count.
  • Docstrings follow Google style, validated by ruff D2,D3,D4 under convention = "google".
  • Describe local logic only.
  • The spec is the only durable document a docstring may reference (docs/spec/... §...). Never reference docs/plans/....
  • A constraint that must guide code long-term belongs in the spec first.
  • No milestone / version references.
  • No PR / task / commit / issue / msg / thread coordinates.
  • No agent or human names; no discussion / review narration.
  • No prose cataloguing what the code does NOT do.
  • A line that genuinely names one of these shapes says hygiene: and why. It excuses that line from the reference checks alone, never from placement, length or width.

Scope

  • A commit touches only what the current task requires.
  • Adjacent / autoformat / submodule changes ship in separate commits.
  • Out-of-scope changes that cannot be split MUST be called out in the commit body.

Spec impact

  • A milestone that changes a public contract lists its owning docs/spec/*.md files in its #### Related Files; one that changes none lists no spec path. There is no separate declaration to keep in step with that list.
  • Contract-changing implementation and its specification ship in the same milestone.

Forward references

  • Break type-only import cycles with quoted forward-reference annotations.
  • Do not guard type-only imports with typing.TYPE_CHECKING; the forward-references pre-commit hook rejects that shim. Ruff cannot be asked to do it — its own TC001-TC003 push the other way.
  • Import runtime dependencies normally or lazily at their point of use.

Tests

  • How a milestone is accepted is settled with the author while the plan is written, in its ##### Accepted by; scripts/finalize_plan_context.py refuses to finalize a plan that leaves that choice open, and prints the contract from docs/policies/project-policy.json.
  • Each milestone defines its own Target State Design before implementation. Show every part the milestone settles in its delivered form as code or compact pseudocode. Implementation and review match that design.
  • A Target State Design covers every step its milestone lists. A step with no delivered shape is a reviewer's finding before implementation begins.
  • One workflow may evidence several acceptance criteria. An acceptance criterion describes behaviour, not a one-test obligation.
  • Lock contracts, not implementation detail. Do not test source text, AST shape, private calls, object identity, counts, or names merely to prevent a future refactor.

DSL / HIR authoring

  • A leading @func docstring is accepted; strings in nested blocks remain bare expressions.
  • Use tf.<op> attribute path; do not alias individual ops.
  • Variadic ops take one explicit list, tuple, or supported static list comprehension; attributes go by keyword.
  • Every @func parameter MUST reach the return through real ops; dead _ = expr assignments do not count.

C++ formatting

  • C++/CUDA sources follow the repo .clang-format (clang-format ≥ 18, LLVM base with IndentWidth: 4, ColumnLimit: 80, PointerAlignment: Right, Standard: c++17). Run pre-commit install once; the versioned .pre-commit-config.yaml then formats every touched *.h/*.hpp/*.cuh/*.cu/*.cpp/*.cc with clang-format -i on commit. An equivalent manual check is clang-format --dry-run -Werror <files>.
  • Formatting scope is touched files only — never run a tree-wide reformat in a feature commit (that belongs in its own isolated commit).
  • Naming: types (struct/class/enum/using aliases) are PascalCase (Topology, Mesh, ShardLayout); functions are snake_case (program_shape, make_shard_tensor); template parameters are T-prefixed PascalCase (TLayout, TMesh, TAttrs) — no trailing underscore.
  • Headers: the umbrella entry header is <tilefoundry/runtime.h>; cuda target headers use .cuh, target-neutral / cpu headers use .h; exactly one TILEFOUNDRY_TARGET_* macro is defined per translation unit and the build injects it per target.