Projects
Production systems, live demos, and open source.
Production AI systems
enterprise scaleSystems designed and operated at enterprise scale inside large engineering organizations. Described at a high level; source is proprietary.
LLM Evaluation Framework
Designed and operated the evaluation framework for generative AI products in production. Defined methodology across retrieval quality (Recall@k, Precision@k, MRR) and generation quality (groundedness, faithfulness), built on Databricks and Spark and adopted as standard practice across AI product teams.
RAG Pipeline Infrastructure
Architected production RAG pipelines connecting LLMs to enterprise knowledge sources. Designed embedding workflows, vector search infrastructure, and retrieval optimization for high-stakes enterprise workloads where hallucination risk is unacceptable.
Agentic Workflow Platform
Designed and deployed agentic AI systems in production: multi-turn orchestration, function calling, and tool-use patterns at enterprise scale across AWS and Azure. Built the infrastructure enabling LLMs to execute multi-step tasks autonomously against internal systems.
Internal Developer Platform
Led the engineering platform serving hundreds of developers across build, ship, and operate workflows. Designed secure-by-default CI/CD pipelines, containerization strategies, and IaC tooling that drove the shift from monolithic deployments to microservices.
Platform reliability & operations
production disciplineSystems and practices ensuring platform-critical infrastructure stays reliable and incident response closes the loop.
Staged Rollout Framework (MAP)
Architected and operated a three-stage onboarding framework (NPE → Canary → Production) governing how Business Units adopt the Moody's Analytics Platform. Clear readiness gates and evaluation criteria ensure platform changes go through structured assessment before affecting all downstream applications, preventing cascade failures.
Observability & Incident Management
Guided enterprise engineering and SRE teams onto Datadog observability stack with PagerDuty integrations. Coordinated incident response and post-mortem ownership with platform core and SRE teams, ensuring a platform-level outage doesn't cascade to every dependent application.
Live demos
running nowRunning systems with public source. You can interact with them or read the code.
edge-ai-agent-lablivemcpworkers-ai
Live Cloudflare Worker implementing the Model Context Protocol (MCP). Exposes three tools (time_now, worker_info, and echo) demonstrating MCP server patterns, Workers AI binding, and edge AI deployment. A reference implementation for agentic tool-use architecture.
TrustClaw28 deploymentsliveclaudejfrogvercel
Autonomous email-summarization agent using Claude Haiku with function calling for tool-based auto-reply. Demonstrates Claude tool-use patterns, serverless deployment to Vercel, and supply-chain auditability through JFrog Artifactory proxying 900+ packages. Includes 5-point DX friction report with actionable fixes.
AI-Vic: conversational AI on the edgeliveworkers-aillama-3.3
The portfolio assistant on this site, grounded in resume, projects, and background. Deployed as a Cloudflare Worker using Workers AI and Llama 3.3 70B: system prompt grounding, edge inference, rate limiting, and conversational UX without a RAG layer. Open the chat widget to try it.
Open source
github.com/ramirez-ai-labsPublic repos covering RAG evaluation, LLM benchmarking, agentic systems, and AI foundations.
RAG Evaluation Labragevalpython
Fully offline, beginner-friendly lab for evaluating RAG systems. Includes synthetic datasets, embeddings, vector search, and retrieval metrics (Recall@k, Precision@k, MRR, and groundedness scoring), built for reproducible RAG evaluation without API dependencies.
AI Operating System (AI-OS)agentsmcplanggraph
Enterprise AI operating system orchestrating multi-agent workflows with Claude as the default LLM provider. Features a 22-case CI-gated evaluation harness running on every commit, a tool-loop safety circuit breaker preventing runaway agent behavior, and strict output validation requiring cited sources or failing at parse time. Demonstrates production patterns for Claude tool-use and agentic safety.
Chatbot Evaluation Systemllm-as-judgeci
Black-box pairwise evaluation baseline for comparing chatbot versions using LLM-as-judge. Supports fake and edge judge paths, audit metadata, and CI integration, with a roadmap toward conversation-level multi-turn evaluation.
OpenAI Foundationsopenaitutorials
Tutorials and demos for building real-world applications with OpenAI APIs, covering function calling, RAG, embeddings, and multimodal applications through four progressive labs from API basics to production patterns.
LATAM GenAI Lakehouse Benchmarklatamsparkdatabricks
Lakehouse-native evaluation framework measuring regional Spanish LLM performance (El Salvador vs Peru) using Delta tables, Spark, and Databricks. Applies Bronze/Silver/Gold data architecture to LLM benchmarking at scale.
Graduate research
UC Berkeley MIDS · 2021 – 2023Projects completed during the UC Berkeley Master of Information and Data Science (MIDS) program.
ML System Engineering & MLOpsmlopskubernetes
End-to-end ML platform built on Kubernetes and microservices, including containerized model serving, automated retraining pipelines, CI/CD for ML, and production monitoring. Stack: Kubernetes, Docker, FastAPI, MLflow.
Machine Learning at Scale: Flight Delay Predictionsparkdatabricks
Distributed ML pipeline predicting flight delays across 30M+ records using MapReduce, Hadoop, and Apache Spark on Databricks. Applied ensemble methods (GBT, Random Forest) with feature engineering on temporal and weather data.
Machine Learning: Understanding Hate Crime Patternstensorflowregression
Applied linear regression and TensorFlow to identify socioeconomic and demographic predictors of hate crime rates across U.S. counties. Surfaced statistically significant correlations to inform policy research.
Data Engineering: Location Recommendations with NoSQLneo4jmongodbredis
Multi-database recommendation engine using Neo4j (graph traversal), MongoDB (document store), and Redis (caching) to generate personalized store location suggestions at low latency.
Data Analysis: NFL Big Data Bowledapandas
Exploratory data analysis on NFL tracking data using Python, NumPy, and Pandas. Analyzed player movement patterns and derived game-level insights from raw positional data.
Statistical Analysis: Movie Revenue Regression Studystatsols
Designed and executed a research study on movie revenue predictors using OLS regression, hypothesis testing, and diagnostic analysis to identify drivers of box office performance.
Data Visualization: Travel Guide Reimaginedtableau
Interactive Tableau dashboard reimagining travel data as a visual guide, layering geographic, seasonal, and sentiment data to surface non-obvious destination insights.
Capstone: enRoute, Running Route Safety Appiosml
iOS app leveraging real-time safety data and ML-based route scoring to recommend safe running routes. Full mobile + backend stack built as UC Berkeley MIDS capstone project.