Claude/develop core concept 011 cuy7 fq zk bn dw hyi q1 m4ub

README.md

@@ -254,6 +254,57 @@ Note that the panorama image must have spherical parameterization (e.g., environ
 The photo is from [this URL](https://commons.wikimedia.org/wiki/Category:360%C2%B0_panoramas_with_equirectangular_projection#/media/File:Braunschweig_Sankt-%C3%84gidien_Panorama_02.jpg)
 </div>
 
+## MoGrammetry: Integrating MoGe with COLMAP for Enhanced 3D Reconstruction
+
+We’ve extended MoGe’s capabilities by combining it with a classical Structure-from-Motion (SfM) pipeline (e.g., COLMAP). This hybrid workflow leverages MoGe’s accurate monocular geometry estimation together with robust camera alignment from COLMAP, enabling faster and more comprehensive 3D reconstructions from image sets.
+
+### Overview
+
+**MoGrammetry** merges single-image 3D point maps generated by MoGe with camera poses and intrinsics recovered by COLMAP. By doing so, we achieve a dense, consistent 3D scene representation without relying solely on multi-view stereo. This pipeline can be particularly beneficial when:
+
+- You have a sequence of images aligned and registered by COLMAP.
+- You want to quickly generate dense and consistent point clouds from each image using MoGe.
+- You aim to integrate those point clouds into a unified 3D model with minimal manual intervention.
+
+### Steps
+
+1. **Run COLMAP or Metashape (COLMAP Export) to Obtain Camera Parameters:**  
+   Use COLMAP (or export from Metashape in COLMAP-compatible format) to compute camera poses and intrinsics.  
+   You should have:
+   - `images.txt` and `cameras.txt` (and optionally `points3d.txt`) in the standard COLMAP format.
+   - A set of aligned and (optionally) converted images.
+
+2. **MoGe Inference per Image:**  
+   For each input image, run MoGe’s `model.infer` to get:
+   - A dense affine-invariant point map.
+   - A mask to filter out sky/undefined geometry.
+   - Intrinsics and scale-invariant depth if needed.
+
+3. **Alignment & Fusion:**
+   - Parse COLMAP’s camera parameters and poses.
+   - Adjust MoGe output to match the COLMAP camera model and transform each set of image-based points into the global coordinate system.
+   - Discard sky and outliers, then merge the resulting point clouds.
+   - Optionally apply outlier removal and meshing techniques for a clean, unified 3D model.
+
+### Example Code
+
+Please refer to the newly added Python script `scripts/colmap_integration.py` in this repository for a working example. The script demonstrates:
+
+- Parsing COLMAP’s `cameras.txt` and `images.txt`.
+- Running MoGe inference on each image.
+- Aligning and merging the resulting point clouds.
+- Saving a final `.ply` file of the reconstructed scene.
+
+### Requirements
+
+In addition to MoGe’s prerequisites, you’ll need:
+- [COLMAP](https://colmap.github.io/) for camera alignment and pose estimation.
+- Open3D or another library for point cloud processing and merging.
+- A compatible Python environment (as described in MoGe’s prerequisites).
+
+By combining MoGe’s single-view geometry with COLMAP’s robust camera alignment, **MoGrammetry** aims to streamline and accelerate your image-based 3D reconstruction workflow.
+
+## License
 See also [`moge/scripts/infer_panorama.py`](moge/scripts/infer_panorama.py)
 
 ## 🏋️‍♂️ Training & Finetuning

examples/config_fast.yaml

@@ -0,0 +1,52 @@
+# Fast MoGrammetry configuration
+# Use for quick previews, lower quality
+
+# Paths (set these before running)
+colmap_model_path: null
+image_dir: null
+output_dir: output/fast
+
+# Model
+model_name: Ruicheng/moge-vitl
+
+# Processing settings
+processing:
+  resolution_level: 6     # Lower quality, faster
+  device: cuda
+  batch_size: 1
+  use_gpu: true
+  save_intermediate: false
+
+# Alignment settings
+alignment:
+  method: least_squares   # Fastest method
+  use_reprojection: false
+  min_valid_points: 100
+
+# Fusion settings
+fusion:
+  voxel_size: 0.02        # Aggressive downsampling
+  outlier_removal: none   # Skip for speed
+  merge_strategy: append  # Fastest merge
+
+# Mesh settings
+mesh:
+  method: ball_pivoting   # Faster than Poisson
+  poisson_depth: 7        # Lower detail
+  ball_pivoting_radii:
+    - 0.01
+    - 0.02
+  simplify_mesh: true
+  target_faces: 100000    # Reduce mesh size
+
+# Output settings
+output:
+  save_point_cloud: true
+  save_mesh: true
+  formats:
+    - ply                 # Only PLY for speed
+  export_report: false
+
+# Logging
+log_level: WARNING        # Less logging
+progress_bar: true

examples/config_quality.yaml

@@ -0,0 +1,63 @@
+# High-quality MoGrammetry configuration
+# Use for best results, slower processing
+
+# Paths (set these before running)
+colmap_model_path: null  # Set to your COLMAP model path
+image_dir: null           # Set to your images directory
+output_dir: output/quality
+
+# Model
+model_name: Ruicheng/moge-vitl
+
+# Processing settings
+processing:
+  resolution_level: 9     # Maximum quality
+  device: cuda
+  batch_size: 1
+  use_gpu: true
+  save_intermediate: false
+
+# Alignment settings
+alignment:
+  method: roe             # Robust Outlier Estimation
+  use_reprojection: true
+  truncation_threshold: 0.03
+  min_valid_points: 200
+  ransac_iterations: 2000
+
+# Fusion settings
+fusion:
+  voxel_size: null        # Auto-determine
+  outlier_removal: both   # Statistical + radius
+  statistical_nb_neighbors: 30
+  statistical_std_ratio: 2.5
+  radius_nb_points: 16
+  radius: 0.05
+  merge_strategy: weighted
+
+# Mesh settings
+mesh:
+  method: poisson
+  poisson_depth: 10       # High detail
+  poisson_width: 0.0
+  poisson_scale: 1.1
+  poisson_linear_fit: false
+  simplify_mesh: false    # Keep full detail
+  target_faces: null
+
+# Output settings
+output:
+  save_point_cloud: true
+  save_mesh: true
+  save_depth_maps: false
+  save_normal_maps: false
+  formats:
+    - ply
+    - glb
+  export_report: true
+
+# Logging
+log_level: INFO
+log_file: null
+progress_bar: true
+verbose: false

examples/example_advanced.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""
+Advanced example with custom processing.
+
+Demonstrates:
+- Loading configuration from file
+- Custom alignment and fusion settings
+- Processing statistics
+- Error handling
+"""
+
+import sys
+from pathlib import Path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from mogrammetry import MoGrammetryPipeline, MoGrammetryConfig
+from mogrammetry.config import AlignmentConfig, FusionConfig, MeshConfig
+import json
+
+
+def main():
+    # Create configuration with custom sub-configs
+    config = MoGrammetryConfig(
+        colmap_model_path='path/to/colmap/sparse/0',
+        image_dir='path/to/images',
+        output_dir='output/advanced_example',
+        model_name='Ruicheng/moge-vitl',
+        log_level='DEBUG',
+        log_file='output/advanced_example/reconstruction.log'
+    )
+
+    # Customize alignment
+    config.alignment = AlignmentConfig(
+        method='roe',
+        use_reprojection=True,
+        truncation_threshold=0.03,
+        min_valid_points=200,
+        ransac_iterations=2000
+    )
+
+    # Customize fusion
+    config.fusion = FusionConfig(
+        voxel_size=0.01,  # 1cm voxels
+        outlier_removal='both',  # Statistical + radius
+        statistical_nb_neighbors=30,
+        statistical_std_ratio=2.5,
+        merge_strategy='weighted'
+    )
+
+    # Customize mesh
+    config.mesh = MeshConfig(
+        method='poisson',
+        poisson_depth=10,  # High quality
+        simplify_mesh=True,
+        target_faces=500000
+    )
+
+    # Save configuration for reference
+    config_path = Path(config.output_dir) / 'config.yaml'
+    config_path.parent.mkdir(parents=True, exist_ok=True)
+    config.to_yaml(str(config_path))
+    print(f"Saved configuration to: {config_path}")
+
+    # Run pipeline with error handling
+    try:
+        print("\nStarting advanced reconstruction...")
+        pipeline = MoGrammetryPipeline(config)
+        stats = pipeline.run()
+
+        # Analyze statistics
+        print("\n" + "=" * 80)
+        print("RECONSTRUCTION STATISTICS")
+        print("=" * 80)
+
+        print(f"\nInput:")
+        print(f"  Total images: {stats['num_images']}")
+        print(f"  Cameras: {stats['num_cameras']}")
+
+        print(f"\nProcessing:")
+        print(f"  Successful: {len(stats['processed_images'])}")
+        print(f"  Failed: {len(stats['failed_images'])}")
+
+        if 'fusion' in stats:
+            fusion = stats['fusion']
+            print(f"\nPoint Cloud Fusion:")
+            print(f"  Input points: {fusion['total_input_points']}")
+            print(f"  After merge: {fusion['points_after_merge']}")
+            if 'points_after_voxel_downsample' in fusion:
+                print(f"  After downsampling: {fusion['points_after_voxel_downsample']}")
+            print(f"  Final points: {fusion['final_point_count']}")
+            if 'total_outliers_removed' in fusion:
+                print(f"  Outliers removed: {fusion['total_outliers_removed']}")
+
+        if 'mesh' in stats and 'error' not in stats['mesh']:
+            mesh = stats['mesh']
+            print(f"\nMesh Generation:")
+            print(f"  Method: {mesh['method']}")
+            print(f"  Vertices: {mesh['vertices_after_cleanup']}")
+            print(f"  Faces: {mesh['faces_after_cleanup']}")
+            if 'faces_after_simplification' in mesh:
+                print(f"  Simplified faces: {mesh['faces_after_simplification']}")
+
+        # Save detailed statistics
+        stats_path = Path(config.output_dir) / 'statistics.json'
+        with open(stats_path, 'w') as f:
+            json.dump(stats, f, indent=2)
+        print(f"\nDetailed statistics saved to: {stats_path}")
+
+        print("\n✓ Reconstruction completed successfully!")
+
+    except Exception as e:
+        print(f"\n✗ Reconstruction failed: {e}")
+        import traceback
+        traceback.print_exc()
+        return 1
+
+    return 0
+
+
+if __name__ == '__main__':
+    exit(main())

examples/example_basic.py

@@ -0,0 +1,46 @@
+#!/usr/bin/env python3
+"""
+Basic example of using MoGrammetry pipeline.
+
+This script demonstrates the simplest way to run a reconstruction.
+"""
+
+import sys
+from pathlib import Path
+
+# Add parent directory to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from mogrammetry import MoGrammetryPipeline, MoGrammetryConfig
+
+
+def main():
+    # Create configuration with required paths
+    config = MoGrammetryConfig(
+        colmap_model_path='path/to/colmap/sparse/0',  # Change this
+        image_dir='path/to/images',                    # Change this
+        output_dir='output/basic_example',
+        model_name='Ruicheng/moge-vitl'
+    )
+
+    # Optional: Customize settings
+    config.processing.resolution_level = 9
+    config.alignment.method = 'roe'
+    config.mesh.method = 'poisson'
+    config.output.save_mesh = True
+    config.output.save_point_cloud = True
+    config.output.formats = ['ply', 'glb']
+
+    # Run pipeline
+    print("Starting MoGrammetry reconstruction...")
+    pipeline = MoGrammetryPipeline(config)
+    stats = pipeline.run()
+
+    # Print results
+    print("\nReconstruction complete!")
+    print(f"Processed {len(stats['processed_images'])} images")
+    print(f"Output directory: {config.output_dir}")
+
+
+if __name__ == '__main__':
+    main()

mogrammetry/__init__.py

@@ -0,0 +1,22 @@
+"""
+MoGrammetry: Integration of MoGe with COLMAP for Enhanced 3D Reconstruction
+
+This package combines MoGe's accurate monocular geometry estimation with COLMAP's
+robust multi-view Structure-from-Motion to create dense, high-quality 3D reconstructions.
+
+Main components:
+- pipeline: Core reconstruction pipeline
+- alignment: Scale and shift alignment solvers
+- fusion: Point cloud fusion and merging
+- mesh: Mesh generation and texturing
+- config: Configuration management
+- utils: Utility functions
+"""
+
+__version__ = "1.0.0"
+__author__ = "MoGrammetry Contributors"
+
+from .pipeline import MoGrammetryPipeline
+from .config import MoGrammetryConfig
+
+__all__ = ['MoGrammetryPipeline', 'MoGrammetryConfig']