Brunosil22/Orena_PROCEDURE_Track_Qwen3_VL_8B_finetuned

ICVS-2Ai ORena FOCUS — PROCEDURE

Two-stage Qwen3-VL-8B system for whole-procedure surgical VQA. No additional annotations were used.

This repository reproduces the latest packaged local PROCEDURE submittion (12 September 2026).

Checkpoints and flow

• segment_base_adapter_step_01800/: SEGMENT initialization merged first into the Qwen base.
• warm2_adapter_step_00800/: PROCEDURE Stage-1 adapter merged after the SEGMENT adapter.
• stage2_adapter/: temporal refiner attached only during Stage 2; all non-temporal questions use Stage 1 alone.
• prepare_merged_model.py: merges the two Stage-1 adapters in the required order.
• requirements.txt: tested Python inference dependencies.
• inference.py: challenge entry point. Stage 1 routes 32, 64, or 128 coarse frames by question family at 768 px. Parseable timestamp questions receive 128 local frames at 768 px from a W3 window computed with a 64-frame coarse reference.
• question_router.py: classifies timestamp templates and allocates coarse samples.
• frame_sampler.py: optional blue-frame/keyframe filtering retained for ablations; disabled in the submittion (CLEAN_FRAMES=0, USE_KEYFRAMES=0).
• resources/system_prompt.txt: exact training and inference prompt.

Prepare and run

Install dependencies and merge the two Stage-1 adapters in order:

python -m pip install -r requirements.txt
python prepare_merged_model.py

stage2_adapter/ remains separate and is enabled only for local temporal refinement. The inference defaults reproduce the packaged candidate configuration:

N_FRAMES=128 IMAGE_MAX_SIDE=768 ROUTED_SAMPLING=1 CLEAN_FRAMES=0 USE_KEYFRAMES=0 \
NF_INSTANT=32 NF_TWO_POINT=64 NF_INTERVAL=128 NF_FORWARD=128 NF_BACKWARD=128 NF_CUMULATIVE=128 \
TWO_STAGE_TEMPORAL=1 TEMPORAL_WINDOW_MULT=3 \
STAGE2_N_FRAMES=128 STAGE2_IMAGE_MAX_SIDE=768 STAGE2_COARSE_REFERENCE_FRAMES=64 \
python inference.py

The explicit variables above document the configuration; they no longer need to be supplied. Set INPUT_PATH, OUTPUT_PATH, or MODEL_DIR only when using non-default locations.

Training code

The shared public training release is available at Brunosil22/orena-focus-training. PROCEDURE Stage 1 uses configs/procedure_warm2_uniform.env, and the temporal refiner uses configs/procedure_stage2.env. These recipes use uniform or deterministic question-routed frames and do not depend on a DINO classifier.