AR Interaction System

• Built a mobile AR application using Unity (AR Foundation with ARKit support)
• Enabled users to:
  • Upload and place 3D models (.glb) in real-world environments
  • Interact with assets through placement, repositioning, and scene control
• Implemented plane detection and surface tracking for stable AR placement

Structured Data Capture

• Designed capture pipeline to extract structured outputs from AR sessions:
  • Background image (camera feed)
  • Foreground mask (binary alpha)
  • HDR environment data (CubeMap / lighting information)
• Ensured captured data is suitable for downstream processing and compositing workflows

Backend Integration

• Integrated with backend services using signed URL uploads
• Designed payload system for transmitting AR session data to processing pipelines
• Enabled asynchronous processing workflows with real-time feedback in the app

Multi-Modal Processing Workflows

• Supported both:
  • 3D workflows → environment-aware processing using captured spatial data
  • 2D workflows → image-based processing using captured frames
• Enabled flexible pipeline integration depending on downstream requirements

Performance & Optimization

• Implemented image compression for faster data transfer
• Optimized model handling for mobile performance constraints
• Ensured stable AR tracking and responsive interaction on supported iOS devices

Use Case & Impact

• Bridges real-world capture with digital content creation pipelines
• Enables scalable data acquisition for AI-driven and rendering workflows
• Supports rapid iteration and content generation from user-driven inputs

‍