# BigGrep Rust - Build and Installation Guide

This guide provides comprehensive instructions for building and installing BigGrep Rust on various platforms.

## Table of Contents

- [Prerequisites](#prerequisites)
- [Quick Start](#quick-start)
- [Platform-Specific Instructions](#platform-specific-instructions)
- [Build Options](#build-options)
- [Installation Methods](#installation-methods)
- [Development Setup](#development-setup)
- [Cross-Compilation](#cross-compilation)
- [Docker Installation](#docker-installation)
- [Performance Tuning](#performance-tuning)
- [Troubleshooting](#troubleshooting)

## Prerequisites

### System Requirements

- **Rust**: 1.70.0 or later
- **Operating System**: Linux, macOS, or Windows
- **Memory**: 2GB+ recommended for building
- **Disk Space**: 1GB+ for build artifacts

### Required Dependencies

#### Linux (Ubuntu/Debian)
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

# Install system dependencies
sudo apt update
sudo apt install -y build-essential pkg-config libssl-dev
```

#### Linux (CentOS/RHEL/Fedora)
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

# Install system dependencies
sudo dnf install -y gcc gcc-c++ openssl-devel pkg-config
# or for older versions:
sudo yum install -y gcc gcc-c++ openssl-devel pkg-config
```

#### macOS
```bash
# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

# Install Xcode command line tools (required for compilation)
xcode-select --install

# Install Homebrew (optional, for additional dependencies)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install openssl pkg-config
```

#### Windows
```powershell
# Install Rust (download from https://forge.rust-lang.org/infra/rustup.html)
# Or use the installer:
iwr -useb https://forge.rust-lang.org/infra/rustup.html | iex

# Install Visual Studio Build Tools or Visual Studio
# Download from: https://visualstudio.microsoft.com/visual-cpp-build-tools/
# Make sure to include "C++ build tools" workload
```

### Optional Dependencies

For enhanced functionality:

#### Archive Support
```bash
# Linux
sudo apt install -y libzip-dev libbz2-dev zlib1g-dev

# macOS
brew install libzip bzip2 zlib

# Windows (included with Rust toolchain)
# No additional installation needed
```

#### Advanced Compression
```bash
# Linux
sudo apt install -y zstd-dev lz4-dev

# macOS
brew install zstd lz4
```

## Quick Start

### Basic Build

```bash
# Clone the repository
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs

# Build all tools in release mode (recommended)
cargo build --release

# Or build for development (faster, but slower at runtime)
cargo build

# Run integration tests
cargo test --release

# Install to default location
cargo install --path crates/biggrep-core
cargo install --path crates/rs-bgindex
cargo install --path crates/rs-bgsearch
cargo install --path crates/rs-bgparse
cargo install --path crates/rs-bgverify
cargo install --path crates/rs-bgextractfile
```

### Verification

```bash
# Verify all tools are installed
rs-bgindex --version
rs-bgsearch --version
rs-bgparse --version
rs-bgverify --version
rs-bgextractfile --version

# Run a quick test
echo "test data" > test.txt
find . -name "test.txt" | rs-bgindex -p test_index
rs-bgsearch -a "test" -d .
```

## Platform-Specific Instructions

### Linux

#### Ubuntu 20.04+ / Debian 11+
```bash
# Install prerequisites
sudo apt update
sudo apt install -y \
    build-essential \
    pkg-config \
    libssl-dev \
    git \
    curl \
    wget \
    unzip

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Build BigGrep
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs

# Build with optimizations for Intel/AMD
cargo build --release --features simd

# Install system-wide
sudo cp target/release/rs-bg* /usr/local/bin/
sudo chmod +x /usr/local/bin/rs-bg*
```

#### CentOS 8+ / Rocky Linux / AlmaLinux
```bash
# Install development tools
sudo dnf groupinstall -y "Development Tools"
sudo dnf install -y \
    openssl-devel \
    pkg-config \
    git \
    curl \
    wget \
    unzip

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Build BigGrep
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs
cargo build --release

# Install to /opt for system-wide access
sudo mkdir -p /opt/biggrep
sudo cp target/release/rs-bg* /opt/biggrep/
sudo ln -sf /opt/biggrep/rs-bg* /usr/local/bin/

# Add to PATH
echo 'export PATH="/opt/biggrep:$PATH"' | sudo tee -a /etc/profile.d/biggrep.sh
```

#### Arch Linux
```bash
# Install base development tools
sudo pacman -S --needed \
    base-devel \
    openssl \
    pkg-config \
    git \
    rustup

# Initialize Rust
rustup default stable

# Build BigGrep
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs
cargo build --release

# Install via AUR (optional)
# Use an AUR helper like yay:
yay -S biggrep-rs
```

### macOS

#### Universal Build (Intel + Apple Silicon)
```bash
# Install Xcode command line tools
xcode-select --install

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Install Homebrew (if not already installed)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Install dependencies
brew install \
    openssl@3 \
    pkg-config

# Build for both architectures
cargo build --release --target x86_64-apple-darwin
cargo build --release --target aarch64-apple-darwin

# Create universal binary (requires additional setup with lipo)
# See cross-compilation section below

# Simple install
cargo install --path crates/rs-bgindex
cargo install --path crates/rs-bgsearch
# ... repeat for other tools
```

#### Intel Macs
```bash
# Build for Intel architecture
cargo build --release --target x86_64-apple-darwin

# Install
cargo install --path crates/rs-bgindex
cargo install --path crates/rs-bgsearch
# ... repeat for other tools
```

#### Apple Silicon (M1/M2/M3)
```bash
# Build for Apple Silicon
cargo build --release --target aarch64-apple-darwin

# Install
cargo install --path crates/rs-bgindex
cargo install --path crates/rs-bgsearch
# ... repeat for other tools
```

### Windows

#### Windows 10/11 (x64)
```powershell
# Install Rust using rustup-init.exe
# Download from: https://forge.rust-lang.org/infra/rustup.html

# Install Visual Studio Build Tools
# Download from: https://visualstudio.microsoft.com/visual-cpp-build-tools/
# Select "C++ build tools" workload

# Open "Developer Command Prompt for VS" (not PowerShell)
# Or run from PowerShell with: 
# & "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

# Clone and build
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs
cargo build --release

# Install to Cargo bin directory
cargo install --path crates/rs-bgindex
cargo install --path crates/rs-bgsearch
# ... repeat for other tools

# Or copy executables manually
Copy-Item target\release\rs-bg*.exe $env:CARGO_HOME\bin\
```

#### Windows on ARM (ARM64)
```powershell
# Note: Limited Rust target support currently
# Consider cross-compilation from x64 Windows

# For native ARM64 development:
# 1. Install Rust for ARM64 (if available)
# 2. Install Visual Studio ARM64 build tools
# 3. Build with: cargo build --release --target aarch64-pc-windows-msvc
```

### FreeBSD
```bash
# Install prerequisites
sudo pkg install -y \
    rust \
    git \
    openssl \
    pkgconf

# Build BigGrep
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs
cargo build --release

# Install
sudo cp target/release/rs-bg* /usr/local/bin/
sudo chmod +x /usr/local/bin/rs-bg*
```

### NetBSD
```bash
# Install Rust via pkgsrc
cd /usr/pkgsrc/lang/rust
sudo make install

# Or install via rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.cargo/env

# Install dependencies
sudo pkg_add openssl pkg-config

# Build and install
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs
cargo build --release
cargo install --path crates/rs-bgindex
# ... repeat for other tools
```

## Build Options

### Release Builds (Recommended)

```bash
# Optimized build for production use
cargo build --release

# With specific features enabled
cargo build --release --features simd,parallel_search

# Profile-guided optimization (advanced)
cargo build --release -Z build-std
# Note: Requires nightly Rust
```

### Debug Builds

```bash
# Faster compilation, slower runtime
cargo build

# With additional debug features
cargo build --features debug_logging,metrics

# Debug build with optimizations disabled
cargo build --profile debug-no-optimize
```

### Specific Target Builds

```bash
# Build specific tool only
cargo build -p rs-bgindex --release
cargo build -p rs-bgsearch --release

# Build with specific features
cargo build -p rs-bgindex --release --features mmapped_io,lockfree

# Build for specific CPU features
RUSTFLAGS="-C target-cpu=native" cargo build --release
```

### Environment Variables

```bash
# Set optimization level
export CARGO_PROFILE_RELEASE_OPT_LEVEL="z"  # Size optimization
export CARGO_PROFILE_RELEASE_OPT_LEVEL="s"  # Speed optimization

# Enable link-time optimization
export RUSTFLAGS="-C lto=fat"

# Set target architecture
export CARGO_BUILD_TARGET=x86_64-unknown-linux-musl

# Parallel compilation jobs
export CARGO_BUILD_JOBS=8
```

## Installation Methods

### Cargo Install (Recommended)

```bash
# Install all tools from source
cargo install --path crates/biggrep-core
cargo install --path crates/rs-bgindex
cargo install --path crates/rs-bgsearch
cargo install --path crates/rs-bgparse
cargo install --path crates/rs-bgverify
cargo install --path crates/rs-bgextractfile

# Install with features
cargo install --path crates/rs-bgindex --features mmapped_io

# Force reinstall
cargo install --path crates/rs-bgindex --force
```

### System Package Manager

#### APT (Debian/Ubuntu)
```bash
# Create Debian package
cargo deb

# Install package
sudo dpkg -i target/debian/biggrep-rs_*.deb
sudo apt-get install -f  # Fix dependencies
```

#### Homebrew (macOS)
```bash
# Create Homebrew formula
cargo create-bottle

# Install via Homebrew
brew install biggrep-rs
```

#### Chocolatey (Windows)
```bash
# Create Chocolatey package
# (Requires additional setup)

# Install
choco install biggrep-rs
```

### Manual Installation

```bash
# Build release version
cargo build --release

# Install to custom location
mkdir -p ~/biggrep/bin
cp target/release/rs-bg* ~/biggrep/bin/
export PATH="$HOME/biggrep/bin:$PATH"

# Or install system-wide
sudo mkdir -p /usr/local/bin
sudo cp target/release/rs-bg* /usr/local/bin/
sudo chmod +x /usr/local/bin/rs-bg*
```

### Docker Installation

See [Docker Installation](#docker-installation) section below.

## Development Setup

### Development Environment

```bash
# Clone repository
git clone https://github.com/your-org/biggrep-rs.git
cd biggrep-rs

# Install Rust toolchain
rustup default stable
rustup component add rustfmt clippy

# Build in debug mode
cargo build

# Run tests
cargo test

# Run integration tests
./integration_tests.sh

# Run benchmarks
cargo bench

# Check formatting
cargo fmt --all -- --check

# Run linter
cargo clippy --all-targets --all-features
```

### IDE Configuration

#### VS Code
```json
{
    "rust-analyzer.linkedProjects": [
        "Cargo.toml",
        "crates/*/Cargo.toml"
    ],
    "rust-analyzer.cargo.features": "all",
    "rust-analyzer.procMacro.enable": true,
    "files.exclude": {
        "**/target": true
    }
}
```

#### CLion
```toml
# .idea/workspace.xml or settings
<component name="CargoSettings">
    <option name="features" value="all" />
    <option name="channel" value="stable" />
</component>
```

### Testing Setup

```bash
# Install test dependencies
cargo test --all-features

# Run specific test categories
cargo test --lib --release  # Unit tests
cargo test --test integration  # Integration tests
cargo test --test benchmarks  # Performance tests

# Run with coverage
cargo install cargo-tarpaulin
cargo tarpaulin --out html

# Run fuzzing tests (requires nightly)
cargo install cargo-fuzz
cargo fuzz run search_fuzzer
```

## Cross-Compilation

### Linux to Windows

```bash
# Install Windows target
rustup target add x86_64-pc-windows-msvc

# Build for Windows
cargo build --release --target x86_64-pc-windows-msvc

# Executables will be in:
# target/x86_64-pc-windows-msvc/release/rs-bg*.exe
```

### Linux to macOS

```bash
# Install macOS targets
rustup target add x86_64-apple-darwin
rustup target add aarch64-apple-darwin

# Build for Intel macOS
cargo build --release --target x86_64-apple-darwin

# Build for Apple Silicon macOS
cargo build --release --target aarch64-apple-darwin
```

### macOS to Linux

```bash
# Install Linux targets
rustup target add x86_64-unknown-linux-gnu
rustup target add x86_64-unknown-linux-musl

# Build for Linux (requires cross-compilation tools)
# On macOS with Homebrew:
brew install FiloSottile/musl-cross/musl-cross

# Build static binary for Linux
cargo build --release --target x86_64-unknown-linux-musl
```

### Universal Binaries (macOS)

```bash
# Build for both Intel and Apple Silicon
cargo build --release --target x86_64-apple-darwin
cargo build --release --target aarch64-apple-darwin

# Create universal binary
lipo -create \
    target/x86_64-apple-darwin/release/rs-bgindex \
    target/aarch64-apple-darwin/release/rs-bgindex \
    -output target/release-universal/rs-bgindex

# Make executable
chmod +x target/release-universal/rs-bgindex
```

## Docker Installation

### Using Pre-built Images

```bash
# Pull latest image
docker pull biggrep-rs:latest

# Run search directly
docker run --rm -v /data:/data:ro biggrep-rs \
    rs-bgsearch -a "pattern" -d /indexes

# Run with interactive shell
docker run -it --rm -v /data:/data:ro biggrep-rs /bin/bash
```

### Building Docker Image

```bash
# Build image from source
docker build -t biggrep-rs:latest .

# Build with specific features
docker build --build-arg FEATURES="simd,mmapped_io" -t biggrep-rs:enhanced .

# Multi-stage build for smaller image
docker build -f Dockerfile.multi-stage -t biggrep-rs:slim .
```

### Docker Compose Setup

```yaml
# docker-compose.yml
version: '3.8'

services:
  biggrep:
    build: .
    volumes:
      - ./data:/data:ro
      - ./indexes:/indexes:rw
    environment:
      - RUST_LOG=info
    command: rs-bgsearch -a "pattern" -d /indexes

  biggrep-search:
    build: .
    volumes:
      - ./data:/data:ro
      - ./indexes:/indexes:rw
    command: tail -f /dev/null  # Keep container running
```

### Dockerfile Examples

#### Standard Dockerfile
```dockerfile
# Dockerfile
FROM rust:1.70-slim as builder

WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY crates ./crates

RUN cargo build --release

FROM debian:bookworm-slim

COPY --from=builder /app/target/release/rs-bg* /usr/local/bin/
RUN chmod +x /usr/local/bin/rs-bg*

ENTRYPOINT ["rs-bgsearch"]
```

#### Multi-stage Dockerfile
```dockerfile
# Dockerfile.multi-stage
FROM rust:1.70-slim as builder

WORKDIR /app
COPY Cargo.toml Cargo.lock ./
COPY crates ./crates
COPY src ./src

# Build with optimizations
ENV RUSTFLAGS="-C lto=fat -C target-cpu=native"
RUN cargo build --release

# Final stage
FROM debian:bookworm-slim

COPY --from=builder /app/target/release/rs-bg* /usr/local/bin/
RUN chmod +x /usr/local/bin/rs-bg*

# Create non-root user
RUN useradd -r -s /bin/false biggrep
USER biggrep

ENTRYPOINT ["rs-bgsearch"]
```

## Performance Tuning

### Build Optimizations

```bash
# Enable Link-Time Optimization
export RUSTFLAGS="-C lto=fat"

# Use native CPU instructions
export RUSTFLAGS="-C target-cpu=native"

# Enable specific CPU features
export RUSTFLAGS="-C target-feature=+avx2,+fma"

# Profile-guided optimization (requires instrumented builds)
cargo build --release -Z build-std
./instrumented_binary --run-workload
cargo build --release -Z build-std
```

### Runtime Optimizations

```bash
# Set number of parallel jobs
export BIGGREP_NUMPROCS=8

# Enable memory mapping
export BIGGREP_USE_MMAP=1

# Set memory limits
export BIGGREP_MAX_MEMORY=2GB

# Enable SIMD instructions
export BIGGREP_SIMD=auto  # auto, enable, disable
```

### System Tuning

```bash
# Increase file descriptor limits
ulimit -n 65536

# Adjust I/O scheduler (Linux)
echo noop > /sys/block/sda/queue/scheduler

# Set swappiness (Linux)
echo 10 > /proc/sys/vm/swappiness

# Disable address space randomization (for reproducible benchmarks)
echo 0 > /proc/sys/kernel/randomize_va_space
```

## Troubleshooting

### Common Build Issues

#### Rust Version Issues
```bash
# Check Rust version
rustc --version

# Update Rust
rustup update stable

# Check for outdated dependencies
cargo outdated
```

#### Compilation Errors
```bash
# Clean build artifacts
cargo clean

# Update dependencies
cargo update

# Check for missing features
cargo tree -p biggrep-core

# Verbose build output
cargo build --release --verbose
```

#### Linker Errors (Linux)
```bash
# Install missing libraries
sudo apt install -y libssl-dev pkg-config

# Or use pkg-config to find libraries
pkg-config --libs openssl
```

#### Linker Errors (macOS)
```bash
# Update Xcode command line tools
sudo xcode-select --reset
xcode-select --install

# Check for missing frameworks
ls -la /usr/local/lib/
```

#### Linker Errors (Windows)
```bash
# Install Visual Studio Build Tools
# Ensure "C++ build tools" workload is selected

# Check for Windows SDK
# Usually installed with Visual Studio

# Verify MSVC environment
"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"
echo %VSCMD_ARG_T_ARCH%
echo %VSCMD_ARG_T_ENV%
```

### Runtime Issues

#### Permission Errors
```bash
# Check file permissions
ls -la target/release/rs-bg*

# Make executable
chmod +x target/release/rs-bg*

# Run with appropriate user
sudo ./target/release/rs-bgindex
```

#### Library Dependencies (Linux)
```bash
# Check shared library dependencies
ldd target/release/rs-bgsearch

# Install missing libraries
sudo apt install -y libssl3 libssl-dev

# Check library paths
echo $LD_LIBRARY_PATH
```

#### Memory Issues
```bash
# Check available memory
free -h

# Monitor memory usage during build
htop

# Build with reduced parallelism
cargo build --release -j 2
```

### Performance Issues

#### Slow Build Times
```bash
# Use incremental compilation
export CARGO_INCREMENTAL=1

# Cache dependencies
export CARGO_HOME=/path/to/cache

# Parallel compilation
export CARGO_BUILD_JOBS=8
```

#### Runtime Performance
```bash
# Check CPU features
lscpu | grep Flags

# Verify SIMD support
cat /proc/cpuinfo | grep avx2

# Monitor system performance
top
iostat -x 1
```

### Getting Help

If you encounter issues not covered here:

1. Check the [GitHub Issues](https://github.com/your-org/biggrep-rs/issues) page
2. Review the [Performance Architecture](PERFORMANCE_ARCHITECTURE.md) document
3. Run the [integration tests](integration_tests.sh) to verify installation
4. Use the [benchmark script](benchmark_comparison.sh) to compare performance
5. Check the main [README](README.md) for additional examples

### Debug Mode

```bash
# Build with debug information
cargo build --profile debug-with-info

# Run with backtraces
RUST_BACKTRACE=1 cargo run --bin rs-bgsearch -- -a "pattern" -d /indexes

# Enable logging
RUST_LOG=debug cargo run --bin rs-bgsearch -- -a "pattern" -d /indexes
```

This comprehensive build and installation guide should help you get BigGrep Rust up and running on any supported platform. For platform-specific issues or advanced configurations, refer to the troubleshooting section or create an issue on the project repository.