🧑 Example of AX from the Employee Perspective - RAG-Based Internal ChatbotExecutive SummaryKey ContributionsAchievementsIntroduction, Problem, and GoalIntroductionProblemGoalTechnical OverviewProblem-Solving in Action: Insights from Overcoming Project Hurdles1. Preventing Data Leakage and Uninformed Use of External LLMs2. Deal with a lot of API Bugs
🧑 Example of AX from the Employee Perspective - RAG-Based Internal Chatbot
Executive Summary
As AI adoption accelerates, organizations are transforming their operations and boosting productivity through advanced digital tools. However, to fully realize these benefits, it is equally important for individual employees to have a foundational understanding of AI and the ability to use these tools effectively in their daily work. This makes it essential to provide a secure, enterprise-grade chatbot that empowers employees to safely leverage AI for internal knowledge and daily tasks.
Key Contributions
- A. Planning & Requirements Discovery:Â Identified organizational pain points and defined requirements for a secure AI knowledge assistant.
- B. LLM Model Integration:Â Designed and implemented a chatbot architecture using the OpenAI API with enterprise-level response control.
- C. UI/UX Development:Â Built a user-friendly web interface enabling seamless use and rapid deployment.
- D. Secure Cloud Deployment:Â Implemented AWS-based auto-scaling infrastructure with monitoring and security policies.
Achievements
- Successfully deployed and operational internal AI chatbot, currently used organization-wide across all branches, including the US, Japan, and other international offices.
- Improved response accuracy by combining ChatGPT capabilities with proprietary knowledge.
Introduction, Problem, and Goal
Introduction
With the rapid advancement of AI, organizations are increasingly seeking to leverage AI and digital transformation to enhance overall productivity and reshape business operations. However, it is equally important for individual employees to possess a foundational understanding of AI and know how to utilize these tools effectively in their daily work.
As AI-based productivity tools began to expand across the organization, employees needed a secure and reliable way to leverage generative AI using internal knowledge sources. However, using public AI services raised security concerns, and internal teams lacked unified access to essential documents and operational knowledge. This led to inefficiencies, slower decision-making, and resource bottlenecks.
To resolve these issues, I planned and deployed a secure RAG-based internal chatbot optimized for enterprise use. The system integrates internal data with the OpenAI API to provide accurate, context-aware responses while maintaining strict access control. Built using Streamlit and deployed on AWS with scalable infrastructure, the chatbot is now actively used by employees to support daily operations and internal inquiries.
Problem
- Employees had limited access to AI tools capable of understanding internal context due to strict security requirements.
- Searching through internal data sources (documents, systems, manuals) was time-consuming and fragmented.
- Public AI models could not be used due to risk of data exposure.
Goal
- Build a secure, internal-use-only RAG-based chatbot to improve knowledge access.
- Deliver accurate domain responses using internal datasets and controlled prompts.
- Reduce dependency on manual inquiries and improve operational efficiency.
Technical Overview
- LLM Integration
- OpenAI (Response API)
- OpenAI Chat Completions & Assistant API (PoC stage)
- Framework & UI
- Streamlit
- Data & DB
- MySQL (internal structured data)
- Infrastructure
- AWS AMI (Amazon Machine Image, for easy updates)
- AWS ASG (Auto Scaling Group, for scalability)
- AWS ALB (Application Load Balancer, for scalability and routing)
- AWS CloudWatch (for logging and monitoring)
Problem-Solving in Action: Insights from Overcoming Project Hurdles
1. Preventing Data Leakage and Uninformed Use of External LLMs
Problem:
Although the solution leverages the OpenAI API, it was not feasible to achieve full on-premise isolation. This introduced a significant technical challenge: how to prevent employees from inadvertently exposing sensitive internal data—such as by uploading confidential documents or failing to opt out of data sharing for model training—while still providing a ChatGPT-like experience.
How I Solved It:
To address this, I implemented strict access controls and user education within the chatbot interface. The system enforces file type and content restrictions, automatically blocks uploads of sensitive data formats, and programmatically ensures that all API calls are made with data usage opt-out parameters set (where supported by OpenAI). Additionally, I integrated clear user warnings and guidance into the UI, making employees aware of data privacy risks and best practices. This combination of technical safeguards and user education minimized the risk of accidental data exposure, even when using a third-party LLM service.
2. Deal with a lot of API Bugs
Problem:
The OpenAI API and related LLM services occasionally returned errors, inconsistent outputs, or failed to handle certain edge cases—especially when dealing with organization-specific queries or large, complex internal datasets.
How I Solved It:
I implemented robust error handling and retry logic at the API integration layer, ensuring that transient failures did not disrupt the user experience. I also developed custom prompt engineering and context management strategies to guide the LLM toward more accurate, relevant answers. By integrating internal datasets and building fallback mechanisms, I ensured that the chatbot could reliably deliver precise, organization-specific information even when the underlying API was unstable.
Â