Incoming Role · Starting Soon

Incoming Founding AI & Data Engineer

Building AI-powered data software: production ML systems, data platforms, and intelligent workflow automation

Starting soon · Ph.D. Computer Engineering · AI Engineering · Data Software · Backend APIs

  • Built AI and data software across 32 production systems serving 30+ school districts — ingestion, ML orchestration, prediction services, compliance intelligence, and workflow automation
  • Turns messy operational data into reliable software products: FastAPI · Go services · SQL Server · MongoDB · Cloudflare Workers · LangGraph · local LLMs
  • Focus: AI-assisted decision systems, data quality, model reliability, idempotent pipelines, observability, and privacy-preserving inference

30+

School Districts

150+

Power BI Dashboards

Rui Bian, PhD — Incoming Founding AI and Data Engineer
0 Systems
Shipped
0+ School
Districts Served
0+ Power BI
Dashboards
0 School Sites
Automated
0K+ California
Schools Mapped
0% Pipeline
Time Saved

System Capabilities

AI & Data Software

AI Product Systems Data Software Platforms Production ML Services FastAPI / Flask Go Services REST APIs LangGraph Local LLMs Data Quality Cloudflare Workers Docker MongoDB SQL Server Idempotent Pipelines Observability

Data Platforms

Batch & Streaming Pipelines ETL Design Data Quality Systems SQL Server / T-SQL Apache Spark Microsoft Fabric pandas / numpy SQLAlchemy Playwright Selenium asyncio SFTP Automation

AI / ML Systems

LangGraph Ollama (Local LLM) OpenAI API Gemini API Predictive Modeling Classification Anomaly Detection Model Evaluation & Monitoring Feature Engineering SHAP Explainability scikit-learn XGBoost ARIMA / Holt-Winters MLflow

Software Stack

Python Go (Golang) TypeScript React / Next.js Tailwind CSS SQL / T-SQL PowerShell Power BI / DAX Streamlit Plotly Mapbox Git

Selected Engineering & Product Work

AI-powered data software and public products — production ML orchestration, data platforms, backend APIs, reliable ingestion services, privacy-first tooling, and operator-facing automation.

01 AI / Data Software Platform

AI Data Workflow Orchestration Platform

ProblemMulti-step data pipelines required manual execution, missed steps, and had no failure diagnosis.
DesignLangGraph state machine with gate-node conditional routing + 4-stage fan-out parallelism; chose local Ollama over cloud LLM — student records never leave the server, eliminating compliance risk at architecture level; per-district fault isolation at node level — single failure triggers LLM diagnosis without cascading; multi-tenant config eliminates per-district code changes.
LangGraphOllamaPythonsubprocessTkinter
75–90% runtime reduction · 30+ districts · 100K+ records/run · node-level fault isolation · zero data egress
02 Go / Data Service

High-Reliability Concurrent SIS Data Ingestion Service

Problem12 SIS dataset types needed reliable ingestion into MongoDB without one school-level API failure aborting a full run.
DesignBuilt a Go service with goroutine fan-out, per-school retry boundaries, dynamic school discovery, startup config validation, MongoDB health checks, and idempotent collection reloads for safe reruns.
GoMongoDBREST APInet/httpConcurrent I/O
12 dataset types · goroutine fan-out · per-school fault isolation · compiled single binary
03 AI Backend API

Adaptive Creative Analysis API

ProblemA creative review workflow needed structured analysis without being locked to one model, one latency tier, or a fragile external provider.
DesignFastAPI service with Pydantic validation, SQLAlchemy persistence, explainable tier routing, OpenAI Structured Outputs, deterministic fallback provider, bounded retries, health checks, and Docker-ready deployment.
FastAPIPydanticSQLAlchemyDockerpytest
Schema-validated API · explainable routing · retry/fallback resilience · auditable failed states
04 Data Ingestion Software

Hybrid API/Automation Government Data Ingestion Pipeline

Problem30+ districts required 45+ minutes of manual portal navigation per cycle to retrieve compliance reports.
DesignDual-mode ingestion uses REST APIs for structured flows and Selenium only for portal-locked reports; automatic LEA-to-credential routing and idempotent truncate-reload keep reruns safe after partial failures.
PythonREST APISeleniumSQLAlchemySQL Server
30+ districts · 7 report types · minimal browser automation surface · idempotent reruns
05 Predictive Data Product

Multi-Domain Risk Prediction & Decision-Support System

ProblemDistricts had no early visibility into student risk across academic, attendance, and behavioral domains.
DesignPer-district model selection, SQL Server batch scoring, OpenAI/Gemini fallback narratives, and idempotent reruns make ML outputs operationally reliable instead of notebook-only.
scikit-learnARIMAHolt-WintersOpenAI APIGemini API
20+ districts · 6 concurrent domains · graceful degradation · idempotent batch scoring
06 Compliance Intelligence

Automated Compliance Monitoring & Risk Scoring System

Problem21 districts tracked federal IEP deadlines manually in spreadsheets, with no automated risk scoring or visibility.
Design4-stage pipeline with SQL extraction, MFA-aware Playwright, PDF parsing, computable compliance rules, district-level fault isolation, and Power BI monitoring.
PlaywrightPDF parsingSQL ServerPower BI
21 districts · 0 missed IEP deadlines · computable-rule compliance · district-level fault isolation
07 Free / Open Source Product

Free Image Tools

ProblemMost online image utilities require uploading files to a server, creating privacy friction for simple compression, conversion, resizing, and PDF workflows.
DesignBuilt a free, open-source, browser-first product where core image processing runs client-side via Web APIs: no upload wait, no server-side file storage, batch downloads, modern WebP/AVIF support, social presets, and AI alt-text generation as an opt-in edge inference workflow.
Client-side JSCanvas APICloudflareSEO
Use free website → View source on GitHub →
Free and open-source image tools · collaboration welcome · compression · conversion · resizing · PDF/image workflows · AI alt text
08 AI Product

OpenChat for AI Agents

ProblemAI agents lack a public, crawlable network where humans and other systems can discover their identities, capabilities, tools, and activity.
DesignBuilt a Threads-inspired, mock-first MVP with eight semantic agent profiles, public post pages, cross-entity search, engagement controls, Supabase OAuth scaffolding, and AI-readable discovery through llms.txt, robots rules, and a generated sitemap. Static export to Cloudflare Workers eliminates request-time server compute.
Next.js 16React 19TypeScriptSupabaseCloudflare
View on GitHub →
Mock-first MVP · 8 agent profiles · semantic public routes · zero request-time compute
09 Open Data Product

California School Explorer

ProblemCalifornia public education data is fragmented across official snapshots, suppressed rows, changing denominators, and context gaps that make school comparison hard for families and researchers.
DesignBuilt an open-source school comparison and trend-analysis platform with transparent methodology, source-aware data processing, same-district and similar-context baselines, subgroup views, shareable comparisons, and coverage/reliability notes instead of opaque rankings.
ReactTypeScriptPythonPostgreSQLCloudflare Workers
View on GitHub →
9,946 public school profiles · 3.9M+ canonical facts · nine indicators · subgroup and similar-context comparison

Experience

Starting Soon
Founding AI and Data Engineer

Incoming role

  • Incoming founding AI/data engineering role focused on production-grade AI systems, backend services, and data infrastructure
  • Start timing: starting soon
  • Brings prior experience shipping multi-tenant automation platforms, ML orchestration, resilient APIs, and public AI-enabled products into AI/data software work
Dec 2022 — Jun 2026
Lead Data Scientist

Expatiate Communications · Pasadena, CA

  • Sole architect and owner of 32 production systems serving 30+ school districts; standardized district onboarding to configuration files — reduced new-district setup from weeks of engineering work to hours of configuration
  • Built LangGraph agentic orchestration achieving 75–90% pipeline runtime reduction; designed 6-domain outcome prediction with per-district model selection; enabled compliance teams to run multi-district risk pipelines without engineering support
  • Deployed operator-first platform on Microsoft Fabric: non-technical staff run complex compliance workflows via GUI launchers; automated Gmail API reporting eliminated manual weekly reporting burden across 18 school sites
2015 — 2022
Ph.D. Researcher — Computer Engineering

University of Delaware · Newark, DE

  • Built internet-scale data collection systems using Python + AWS, processing millions of network probes for BGP routing and proxy ecosystem analysis
  • Published at IEEE INFOCOM 2024, Elsevier Computer Networks 2022, ACM SIGCOMM CCR 2019 — GPA 3.96 / 4.0
  • TPC member and reviewer: IEEE INFOCOM, DSN, TNSE, Computer Networks
Prior Experience
Engineering & Research Roles

7+ years across research and industry positions

  • Broad engineering background spanning systems, data, and applied research prior to doctoral studies
  • Master research in Engineering at University of Science and Technology of China (USTC)
  • Research internship at Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences
  • B.S. in Engineering — University of Science and Technology of China (USTC)

Platform Design Principles

Three non-negotiable engineering constraints applied across all 32 production systems — not policies, but structural guarantees.

Fault Isolation by Default

Every system isolates failures at the tenant boundary. One school or district failing never cascades to others — enforced at the goroutine or node level, not by try/catch wrapping.

  • Per-district error recovery node in LangGraph pipelines — failure triggers LLM diagnosis, not pipeline abort
  • Per-goroutine isolation in Go SIS Service — individual timeouts logged and skipped without blocking concurrent streams
  • Single MFA failure never aborts IEP Compliance Pipeline across 21 districts
  • Pre-flight Extract_checker validates all inputs before any database write

Idempotent Operations

All ETL pipelines use truncate-reload semantics — never append-only. Any pipeline is safe to rerun after partial failure without data corruption, duplication, or manual cleanup.

  • Truncate-reload in CALPADS Pipeline — consistent state guaranteed regardless of failure point
  • Idempotent batch scoring in Risk Prediction System — results reproducible given same input snapshot
  • 30+ districts on shared infrastructure with strict tenant data isolation
  • Test mode in every user-facing system — no accidental production sends or overwrites

Privacy by Architecture

AI inference on student data runs locally via Ollama — a structural constraint, not a configuration option. Eliminates PII exposure risk regardless of downstream code changes or vendor policy.

  • Local Ollama qwen2.5:7b for failure analysis — zero student data sent to cloud LLM APIs
  • No student PII in any external API payload — enforced by design, not by access control policy
  • Multi-tenant architecture: district A cannot access district B data by construction
  • Structured audit log maintained locally for all pipeline runs

Engineering Philosophy

Reliability is a product feature. Systems handling public-sector student data must remain correct under partial failure, degraded upstream inputs, and operational retries. Design approach prioritizes isolation boundaries, deterministic recovery semantics, and operational transparency — not as optimizations, but as first-class requirements that shape every architectural decision from the start.
Systems scale humans, not just compute. Every system I build is operated by non-technical staff: compliance coordinators, district program managers, school administrators. Architectural decisions account for the human operating layer — test modes before production sends, color-coded risk signals instead of raw scores, single-command automation for workflows that previously required engineering involvement. The measure of leverage is not pipeline throughput, but whether your systems enable people who couldn't do the work before to do it reliably now.

High-Impact Engineering

Transforming EdTech Intelligence at Scale

Expatiate Communications Lead Data Scientist

The Challenge

School districts lacked predictive visibility into IEP compliance, academic progress, and operational risk — relying on slow, fragmented manual data aggregation across disparate assessment platforms.

Architecture

  • Designed a LangGraph agentic pipeline orchestrator with gate-node conditional routing, fan-out parallelism across 4 concurrent stages, and a local Ollama LLM (qwen2.5:7b) that analyzes failures and suggests fixes — no student data leaves the server
  • Built a 6-domain ML prediction system (CAASPP ELA/Math, ELPAC, Chronic Absenteeism, College/Career, Suspension) using scikit-learn, ARIMA, and Holt-Winters with per-district model selection; integrated OpenAI + Gemini APIs for plain-language administrative narratives
  • Automated IEP compliance tracking across 21 districts: Playwright PDF download with MFA handling → PDF date parsing → green/yellow/orange/red deadline risk scoring → Power BI dashboards; zero missed IEP deadlines after deployment
  • Engineered 32 production tools covering ETL, compliance reporting, browser automation (Playwright + Selenium), async web scraping, SFTP delivery, a Go REST API client to MongoDB, and 150+ Power BI dashboards across 9 dashboard types
  • Built automated Gmail API email alert system delivering data-driven weekly performance summaries to 18 school sites with dynamic metric selection from 12 indicators

Business Impact

Platform deployed across 30+ California school districts. LangGraph agentic automation achieved a 75–90% reduction in pipeline processing time. Replaced dozens of hours of weekly manual work — data collection, compliance tracking, report generation, and stakeholder communication — with single-command automated pipelines.

Internet-Scale Transparent Proxy Analysis

University of Delaware Ph.D. Research · IEEE INFOCOM 2024

The Challenge

Transparent proxies silently intercept and modify web traffic without user awareness, but their true prevalence, behavior, and network-wide impact were poorly understood at scale.

Methodology

  • Designed a large-scale active measurement system to detect and fingerprint transparent proxies across global internet paths
  • Built Python-based data collection and analysis pipelines processing millions of network probes
  • Developed novel detection heuristics combining HTTP header analysis and TCP-level signals

Academic Impact

Published at IEEE INFOCOM 2024 — one of the top-ranked venues in computer networking, revealing the significant hidden influence of transparent proxies on internet traffic integrity.

Mapping the Open Proxy Ecosystem

University of Delaware Ph.D. Research · Computer Networks 2022

The Challenge

The open proxy landscape — used for anonymization, censorship circumvention, and malicious activity — had never been comprehensively characterized in terms of scale, geography, and behavior.

Methodology

  • Crawled, scanned, and analyzed 436,000+ open proxies across the global internet
  • Built large-scale data collection infrastructure using Python and AWS for distributed scanning
  • Applied statistical modeling and traffic analysis to characterize proxy behavior, uptime, and abuse patterns

Academic Impact

Published in Computer Networks (Elsevier), 2022 — delivering the first comprehensive analysis of the open proxy ecosystem and its security implications at internet scale.

Anycast Routing & Remote Peering Effects

University of Delaware Ph.D. Research · ACM SIGCOMM CCR 2019

The Challenge

Remote peering in BGP networks was known to distort anycast routing decisions, but the extent of this unintended impact on global traffic distribution — including for major cloud providers — had not been passively quantified.

Methodology

  • Developed a passive BGP measurement methodology to infer anycast catchment boundaries without active probing
  • Analyzed global BGP routing tables and AS-path data across hundreds of vantage points
  • Correlated routing anomalies with remote peering relationships at internet exchange points (IXPs)

Academic Impact

Published in ACM SIGCOMM Computer Communication Review, 2019 — a flagship networking venue — establishing foundational methodology for passive anycast analysis used in subsequent internet measurement research.

In Progress

AI Cloaking & Content Differentiation on the Open Web

Independent Research Target: IMC / WWW / USENIX Security

The Challenge

As AI crawlers become ubiquitous, websites are moving beyond binary blocking (robots.txt) to a more sophisticated, unmeasured tactic: returning HTTP 200 responses to both humans and AI bots, but serving degraded, watermarked, or "poisoned" content specifically to crawlers like GPTBot.

Methodology

  • Twin-crawler framework (Playwright) visiting Tranco Top 10,000 domains — once as a standard browser UA, once as GPTBot
  • DOM tree structural comparison and text similarity scoring via Jaccard & TF-IDF cosine similarity
  • Sector-level taxonomy: paywall injection, text truncation, gibberish poisoning, visual watermarking

Novelty

Unlike prior work measuring blocking, this measures deception — filling a critical gap in understanding how the web's content landscape diverges between human and AI readers.

In Progress

LLM-Hallucinated Infrastructure Domains as an Attack Surface

Independent Research Target: NDSS / CCS / USENIX Security

The Challenge

LLMs are widely used to generate Infrastructure-as-Code (Terraform, Kubernetes YAML, Nginx configs). If a model hallucinates a plausible but unregistered domain endpoint, an attacker could register that domain to intercept live API traffic or credentials from deployed systems.

Methodology

  • 1,000+ DevOps-focused prompts submitted to GPT-4o, Claude 3.5 Sonnet, and Llama-3-70B
  • Regex extraction of all generated domains, filtered against known public registries
  • DNS resolution + Registrar API queries to quantify hallucination rate and live registrability of phantom endpoints

Novelty

Distinct from package hallucination studies — this targets DNS-level infrastructure interception, a critical supply chain risk not previously measured in the LLM security literature.

Professional Credentials

DC

DataCamp

4 Active Certifications · Issued 2026 · Valid through 2028

AI Engineer for Developers Associate
AI Engineer for Data Scientists Associate
Data Scientist Associate
Data Engineer Associate
G

Google Cybersecurity Professional Certificate

Coursera · Issued Aug 2023

Thought Leadership

Selected Publications & Patents

Silent Observers Make a Difference: A Large-scale Analysis of Transparent Proxies on the Internet.

Rui Bian et al. | IEEE INFOCOM, 2024.

Shining a Light on Dark Places: A Comprehensive Analysis of Open Proxy Ecosystem.

Rui Bian et al. | Computer Networks, 2022.

Towards Passive Analysis of Anycast in Global Routing: Unintended Impact of Remote Peering.

Rui Bian et al. | ACM SIGCOMM CCR, 2019.

Patent: Manufacturing method of micro lens / 一种微透镜的制作方法.

Gang Liu, Ying Xiong, Rui Bian et al. | CN104614936B.

Academic Service

Extensive peer review contributions ensuring the integrity and quality of high-tier network science and security venues.

Key TPC / Reviewer Roles:

  • IEEE INFOCOM ('17, '18, '19, '20, '21)
  • IEEE/IFIP DSN ('19, '21, '22)
  • IEEE Transactions on Network Science and Engineering (TNSE)
  • Computer Networks
  • IEEE ITEC, IEEE RTC, IEEE SmartSys

Professional Recommendations

LinkedIn recommendations from an academic advisor, teammates, and leadership who worked directly with Rui on research, production software, AI/ML, and data platform work.

July 8, 2026
Haining Wang Professor at Virginia Tech PhD co-advisor; managed Rui directly

I highly recommend Dr. Rui Bian for any IT or Cybersecurity role. As his PhD co-advisor, I witnessed his exceptional analytical skills and relentless attention to detail firsthand. Rui didn't just study cybersecurity; he actively solved complex problems in different IT domains, such as backend services, infrastructure automation, and production ML platforms.

Beyond his technical expertise, Rui is a methodical researcher and an outstanding communicator. He knows exactly how to document and articulate complex IT tasks to both technical teams and non-technical stakeholders. Any IT team would be lucky to have him on board. I strongly endorse him for his next career step.

June 15, 2026
Zihao (Peter) Xu Software Engineer @ Apple | USC Alum Worked with Rui on the same team

Having worked with Rui on Expatiate Communication's data team, I can confidently say that he has been an exceptional leader and a key contributor to the team. He has deep technical expertise in AI/ML, data science, and software engineering, and has successfully led the team to deliver impactful products like iTAAP, the company's core data-driven, AI/ML-powered platform.

Rui is also a proactive learner. I know he has earned multiple data science certifications in his spare time and is able to quickly apply what he learns to real-world projects. For example, when LLMs started gaining popularity in 2023, he took the initiative to integrate them into the company's products, leading to one of the first LLM-powered solutions in the education technology market.

Overall, I strongly believe Rui will succeed wherever he goes because of his solid professional skills, technical vision, and drive to keep learning and innovating.

June 14, 2026
Shaonan (Sarah) Hua Software Engineer II Worked with Rui on the same team

I had the pleasure of working with Rui and was consistently impressed by his technical depth and ability to deliver impactful solutions. His expertise spans the full ML lifecycle, from data engineering and predictive modeling to deployment and operationalization. He combines a strong research foundation with practical engineering skills, enabling him to solve complex problems with rigor and efficiency.

I highly recommend him to any organization looking for a talented Applied Scientist or Data Scientist who can drive both innovation and business impact.

June 13, 2026
Maged Marcus IT Director | Technology Services & Operations Leader Managed Rui directly

I had the pleasure of working with Rui Bian on my team, and I can say without hesitation that he is one of the most technically strong and self-driven engineers I've had the opportunity to manage.

Rui consistently demonstrated deep expertise across both engineering and data science, whether he was architecting scalable systems, building robust data pipelines, or applying statistical and ML techniques to solve complex problems. What set him apart wasn't just his technical skill, but his ability to translate that skill into real, measurable impact for the team and the business.

One of Rui's most defining qualities is his sense of ownership. He never waited to be told what needed to be done. He identified problems, proposed solutions, and drove them to completion with minimal hand-holding. When things got complicated or ambiguous, Rui leaned in rather than backing away. His problem-solving approach is methodical yet creative, and he has a rare ability to cut through complexity and find elegant, practical solutions.

Beyond the technical contributions, Rui made the team better. His rigor raised the bar for code quality and analytical thinking, and his collaborative approach made him someone others genuinely wanted to work with.

Let's Connect

Incoming Founding AI and Data Engineer, starting soon. Focused on AI-powered data software, production ML systems, and backend data platforms. Open to technical conversations, research collaboration, and product-building connections.

Email LinkedIn GitHub
Resume

  Loading GitHub activity…

Send a Message

Ask Rui's AI Powered by Llama 3.1

Rui's AI Assistant

Powered by Llama 3.1 · Ask anything
Hi! I'm Rui's AI assistant. Ask me anything about his skills, projects, or experience — I'll give you a straight answer.