tawasuyu

sergio/takana

> Una distribución Linux construida con un laboratorio funcional y hermético en el > sótano, y una terminal mutable, clásica y anárquica en el piso de arriba — diseñada > desde el suelo para que una IA programadora entienda, modifique y co…

TOML 1572Shell 203Rust 109Markdown 993728 Definiciones

selfhost-verify: artefactos del frente rust (parche run_rustc + runners stage0-musl)

Rastrea las correcciones del frente rust que vivían en .scratch/ (gitignored):

- run_rustc.patch (vs mrustc be69c74): minicargo/hello con --target musl;
  RUSTFLAGS + sitios de link con link-self-contained=no (CRT vía gcc Alpine,
  link dinámico contra musl); proxy host-units con crt-static off (sin inject
  de --target, host==target==musl con el stage0 musl-host).
- build-stage0-musl.sh: rebuild del stage0 rustc+cargo con host triple musl
  (CFG_COMPILER_HOST_TRIPLE vía RUSTC_TARGET, --target vía MRUSTC_TARGET),
  reusa LLVM + bin/mrustc. Gotcha OUTDIR_SUF documentado.
- run-runrustc.sh: corre targets de run_rustc en el sandbox devfs apuntando
  al stage0 musl-host.
- README: diagnóstico de los 3 snags acoplados (std-gnu mmap64/open64,
  CRT self-contained ausente, host-triple gnu + proc-macros) y el workflow.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

bf3f1166ec97d80109c53af44e1de58d06ed8aff · por sergio · (hace 4 meses) · padre fc8bc906e · ver el árbol en este commit

Delta semántico

Ninguna definición cambió: el cambio es de texto, documentación o datos.

4 archivos · +191 −0

diff --git a/scripts/rust-frontier/README.md b/scripts/rust-frontier/README.md
new file mode 100644
index 00000000..a70be831
--- /dev/null
+++ b/scripts/rust-frontier/README.md
@@ -0,0 +1,80 @@
+# rust-frontier — self-hosting the builder's rust toolchain
+
+The last frontier of `selfhost-verify` variante (b): replace the Alpine
+`rustc/cargo 1.91.1` in `/toolchain` with a hammer-built rust, via the **purist
+mrustc chain** (no prebuilt rustc binary). Path:
+
+    mrustc → rustc 1.90.0 (builds bundled LLVM) → 1.91.0 → 1.91.1
+
+These scripts + patch are the working artifacts. The mrustc tree itself lives in
+`.scratch/mrustc` (gitignored — it's a large third-party clone, upstream master
+`be69c74`); only our deltas are tracked here.
+
+## Files
+
+- **`run_rustc.patch`** — patch against mrustc `be69c74` for `run_rustc/Makefile`
+  and `run_rustc/rustc_proxy.sh`. Apply from the mrustc root:
+  `git -C .scratch/mrustc apply /path/to/run_rustc.patch`
+- **`build-stage0-musl.sh`** — rebuild the mrustc stage0 `rustc`+`cargo` with a
+  **musl host triple** (`output-1.90.0-x86_64-unknown-linux-musl/`). Run inside
+  the Alpine devfs sandbox (gcc, MRUSTC_TARGET_VER=1.90). Reuses the already-built
+  LLVM and `bin/mrustc`.
+- **`run-runrustc.sh`** — run a `run_rustc` make target inside the devfs sandbox,
+  pointed at the musl-host stage0.
+
+> Both runners hardcode `/home/sergio/hammer` and assume the prepared devfs at
+> `.dev-fs/alpine` (256-TLS-key musl loader + g++; see the bootstrap-devfs notes).
+
+## Why the patches exist (the snags)
+
+The lab is **musl-only** (Alpine devfs); the 4/4 inputs (arje-zero/hammerd) are
+built `--target x86_64-unknown-linux-musl`. Three coupled problems had to be fixed
+to bootstrap a usable rust there:
+
+1. **stage-1 std built for gnu** (`mmap64`/`open64` undefined at link).
+   `minicargo` was invoked without `--target`, so it used mrustc's default **gnu**
+   triple — and gnu std references glibc's LFS64 symbols that musl 1.2.5 doesn't
+   export. Fix: pass `--target $(RUSTC_TARGET)` to minicargo (Makefile:162) and to
+   the stage-1 smoke-test (Makefile:172). A correct musl std uses the **base** libc
+   symbols (`open`/`mmap`/…), matching Alpine's reference std.
+
+2. **self-contained CRT absent.** musl's default link is `-static-pie` against a
+   `lib/rustlib/<t>/lib/self-contained/` CRT dir that the mrustc sysroot never
+   populates. Fix: `-C target-feature=-crt-static -C link-self-contained=no` so the
+   Alpine gcc supplies crt1/crti/crtn + crtbeginS/crtendS and links dynamically
+   against the system musl (RUSTFLAGS in Makefile:111, plus the per-link sites).
+
+3. **host-triple = gnu on a musl system** (the deep one). cargo compiles
+   build-scripts and **proc-macros** for the *host*; mrustc's stage0 rustc has its
+   host triple baked as `x86_64-unknown-linux-gnu` (not runtime-overridable). A
+   proxy `--target musl` inject fixes build-script *bins*, but **proc-macros are
+   dlopen'd and must match the host triple structurally** — injection makes them
+   target artifacts that rustc rejects. Aliasing musl rlibs as gnu also fails:
+   rustc reads the triple from rlib *metadata* (`E0461`), not the path.
+
+   **The only fix is a stage0 rustc whose host triple is musl** — that's what
+   `build-stage0-musl.sh` produces (`CFG_COMPILER_HOST_TRIPLE` via `RUSTC_TARGET`,
+   `--target` via `MRUSTC_TARGET`). Then host==target==musl: cargo doesn't
+   cross-compile, proc-macros load natively, and `rustc_proxy.sh` only needs the
+   CRT flags (no `--target` inject).
+
+   Gotcha: do **not** pass `OUTDIR_SUF` on make's command line — a command-line var
+   can't be reassigned by the makefile, so the `-musl` suffix append is skipped and
+   the build contaminates the gnu output dir. Let it default.
+
+## Workflow
+
+```sh
+# 1. apply the run_rustc patch into the mrustc clone
+git -C .scratch/mrustc apply scripts/rust-frontier/run_rustc.patch
+
+# 2. rebuild stage0 rustc+cargo with a musl host triple (~hours, reuses LLVM)
+.scratch/build-stage0-musl.sh            # -> output-1.90.0-x86_64-unknown-linux-musl/
+
+# 3. build the optimised rustc + cargo via run_rustc (stage 1/2/3)
+.scratch/run-runrustc.sh all
+
+# 4. (next) climb 1.90.0 → 1.91.0 → 1.91.1, then swap rust into /toolchain
+```
+
+Full diagnostic history is in the `selfhost-verify-rust-frontier` memory.
diff --git a/scripts/rust-frontier/build-stage0-musl.sh b/scripts/rust-frontier/build-stage0-musl.sh
new file mode 100755
index 00000000..93c4ab26
--- /dev/null
+++ b/scripts/rust-frontier/build-stage0-musl.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+# Path A: rebuild the mrustc stage0 rustc+cargo with a MUSL host triple, so
+# CFG_COMPILER_HOST_TRIPLE=x86_64-unknown-linux-musl is baked in. This lets
+# run_rustc's cargo build proc-macros/build-scripts natively (host==target==musl)
+# without the proxy --target hack. LLVM + mrustc are reused (already built).
+# Output lands in output-1.90.0-x86_64-unknown-linux-musl/.
+set -eu
+ROOT=/home/sergio/hammer
+DEVFS=$ROOT/.dev-fs/alpine
+MRUSTC=$ROOT/.scratch/mrustc
+T=x86_64-unknown-linux-musl
+
+run() {
+  bwrap \
+    --bind "$DEVFS" / \
+    --bind "$MRUSTC" /mrustc \
+    --dev /dev --proc /proc \
+    --tmpfs /tmp --tmpfs /run \
+    --setenv PATH /usr/bin:/bin:/usr/sbin:/sbin \
+    --setenv HOME /root --setenv TMPDIR /tmp \
+    --setenv MRUSTC_TARGET_VER 1.90 \
+    --setenv CC gcc --setenv CXX g++ \
+    --chdir /mrustc \
+    make -f minicargo.mk \
+      RUSTC_VERSION=1.90.0 \
+      MRUSTC_TARGET=$T RUSTC_TARGET=$T \
+      RUSTC_INSTALL_BINDIR=bin PARLEVEL=8 \
+      "$@"
+}
+
+echo "=== [1/3] LIBS (musl bootstrap std) ==="
+run LIBS
+echo "=== [2/3] rustc (musl host) ==="
+run output-1.90.0-$T/rustc
+echo "=== [3/3] cargo (musl host) ==="
+LIBGIT2_SYS_USE_PKG_CONFIG=1 run output-1.90.0-$T/cargo
+echo "=== DONE ==="
+ls -la output-1.90.0-$T/rustc output-1.90.0-$T/cargo
diff --git a/scripts/rust-frontier/run-runrustc.sh b/scripts/rust-frontier/run-runrustc.sh
new file mode 100755
index 00000000..f7d7598b
--- /dev/null
+++ b/scripts/rust-frontier/run-runrustc.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+# Run a run_rustc make target inside the Alpine devfs sandbox (musl, 256-TLS loader, gcc).
+# Usage: run-runrustc.sh <make-target> [extra make args...]
+set -eu
+ROOT=/home/sergio/hammer
+DEVFS=$ROOT/.dev-fs/alpine
+MRUSTC=$ROOT/.scratch/mrustc
+TARGET=${1:?make target required}
+shift || true
+
+exec bwrap \
+  --bind "$DEVFS" / \
+  --bind "$MRUSTC" /mrustc \
+  --dev /dev --proc /proc \
+  --tmpfs /tmp --tmpfs /run \
+  --setenv PATH /usr/bin:/bin:/usr/sbin:/sbin \
+  --setenv HOME /root \
+  --setenv TMPDIR /tmp \
+  --setenv MRUSTC_TARGET_VER 1.90 \
+  --setenv CC gcc --setenv CXX g++ \
+  --chdir /mrustc/run_rustc \
+  make RUSTC_VERSION=1.90.0 OUTDIR_SUF=-1.90.0-x86_64-unknown-linux-musl \
+       RUSTC_TARGET=x86_64-unknown-linux-musl PARLEVEL=8 \
+       "$TARGET" "$@"
diff --git a/scripts/rust-frontier/run_rustc.patch b/scripts/rust-frontier/run_rustc.patch
new file mode 100644
index 00000000..2e503132
--- /dev/null
+++ b/scripts/rust-frontier/run_rustc.patch
@@ -0,0 +1,49 @@
+diff --git a/run_rustc/Makefile b/run_rustc/Makefile
+index e78d8c5..4d6ef5d 100644
+--- a/run_rustc/Makefile
++++ b/run_rustc/Makefile
+@@ -108,7 +108,7 @@ CARGO_FLAGS := --target $(RUSTC_TARGET) -j $(PARLEVEL) --release --verbose
+ CARGO_ENV := CFG_COMPILER_HOST_TRIPLE=$(RUSTC_TARGET)
+ CARGO_ENV += RUSTC_BOOTSTRAP=1
+ CARGO_ENV += CARGO_HOME=$(CARGO_HOME)
+-CARGO_ENV += RUSTFLAGS="-Z force-unstable-if-unmarked -C link_args=-Wl,-rpath,\$$ORIGIN/../lib -C target-feature=-crt-static"
++CARGO_ENV += RUSTFLAGS="-Z force-unstable-if-unmarked -C link_args=-Wl,-rpath,\$$ORIGIN/../lib -C target-feature=-crt-static -C link-self-contained=no"
+ 
+ fn_opt_copy = (test -e "$2" && test ! "$1" -nt "$2") || cp "$1" "$2"
+ 
+@@ -159,7 +159,7 @@ $(LIBDIR_S)libstd.rlib: $(BINDIR_S)rustc $(MINICARGO) Makefile
+ 	@echo "[MINICARGO] $(RUST_SRC_LIBS)std > $(LIBDIR_S)"
+ 	mkdir -p $(LIBDIR_S)
+ ifneq ($(TARGETVER_LEAST_1_74),)
+-	$V+STD_ENV_ARCH=$(RUSTC_ARCH) MRUSTC_PATH=$(abspath $(BINDIR_S)rustc) $(MINICARGO) --vendor-dir $(VENDOR_DIR) --script-overrides ../script-overrides/stable-$(RUSTC_VERSION)-$(PLATFORM)/ --output-dir $(LIBDIR_S) $(RUST_SRC_LIBS)sysroot
++	$V+STD_ENV_ARCH=$(RUSTC_ARCH) MRUSTC_PATH=$(abspath $(BINDIR_S)rustc) $(MINICARGO) --vendor-dir $(VENDOR_DIR) --script-overrides ../script-overrides/stable-$(RUSTC_VERSION)-$(PLATFORM)/ --output-dir $(LIBDIR_S) --target $(RUSTC_TARGET) $(RUST_SRC_LIBS)sysroot
+ else
+ 	$V+MRUSTC_PATH=$(abspath $(BINDIR_S)rustc) $(MINICARGO) --vendor-dir $(VENDOR_DIR) --script-overrides ../script-overrides/stable-$(RUSTC_VERSION)-$(PLATFORM)/ --output-dir $(LIBDIR_S) $(RUST_SRC_LIBS)std
+ 	$V+MRUSTC_PATH=$(abspath $(BINDIR_S)rustc) $(MINICARGO) --vendor-dir $(VENDOR_DIR) --script-overrides ../script-overrides/stable-$(RUSTC_VERSION)-$(PLATFORM)/ --output-dir $(LIBDIR_S) $(RUST_SRC_LIBS)panic_unwind
+@@ -169,7 +169,7 @@ endif
+ # Build hello_world using the bootstrapping compiler (mrustc-built rustc)
+ $(BINDIR_S)hello_world: $(RUST_SRC_HELLO) $(LIBDIR_S)libstd.rlib $(BINDIR_S)rustc
+ 	@mkdir -p $(dir $@)
+-	$V$(BINDIR_S)rustc $(RUSTFLAGS_$@) -L $(LIBDIR_S) $< -o $@
++	$V$(BINDIR_S)rustc $(RUSTFLAGS_$@) --target $(RUSTC_TARGET) -C target-feature=-crt-static -C link-self-contained=no -L $(LIBDIR_S) $< -o $@
+ 
+ # ---
+ # Stage 2: Build a proper libstd (which includes dynamc libraries too)
+diff --git a/run_rustc/rustc_proxy.sh b/run_rustc/rustc_proxy.sh
+index 3f4f2b9..78f6f1d 100755
+--- a/run_rustc/rustc_proxy.sh
++++ b/run_rustc/rustc_proxy.sh
+@@ -17,7 +17,12 @@ if is_target "$@"; then
+     #echo "  [REAL]" "$@" >&2
+     ${PROXY_RUSTC} "$@"
+ else
++    # Host (build-script / proc-macro) build. With the musl-host stage0 rustc its
++    # default target IS x86_64-unknown-linux-musl, so no --target inject is needed
++    # (host==target==musl, proc-macros load natively). Just force dynamic +
++    # non-self-contained so the Alpine gcc supplies the CRT (the mrustc sysroot has
++    # no self-contained CRT dir, and musl's default is static-pie).
+     #echo "  [BOOTSTRAP]" "$@" >&2
+-    ${PROXY_MRUSTC} "$@"
++    ${PROXY_MRUSTC} -C target-feature=-crt-static -C link-self-contained=no "$@"
+ fi
+