Cloud DevOps Engineer

Document Question Answering with Generative AI and Elasticsearch

This project provides a hands-on demonstration of a DevOps workflow designed to automate and test local development environments with seamless integration into cloud-based DevOps processes. The focus lies on functional and integration testing specifically involving Elasticsearch and Ollama, rather than isolated unit testing.

The `llmdoc` Python application employs a Retrieval-Augmented Generation (RAG) approach. It leverages a Large Language Model (LLM) to summarize search results retrieved from Elasticsearch document queries.

Key Components:

spaCy (https://spacy.io/) library for text analysis tasks

Elasticsearch for document indexing and search

Ollama Large Language Model inference API

GitHub

Google Cloud Infrastructure for Document Question Answering

Document Question Answering System using Generative AI and Elasticsearch on Google Cloud

Key Components:

Google Cloud Platform (GCP) for infrastructure hosting

Google Kubernetes Engine (GKE) for container orchestration

HELM for packaging and deploying Kubernetes applications

Terraform for infrastructure as code and provisioning

HashiCorp Vault for secrets management

Static code analysis with Checkov (https://www.checkov.io/)

GitHub

HashiCorp Vault deployment into Google Kubernetes Engine (GKE)

The project implements Continuous Delivery (CD) of Hashicorp Vault into a private GCP Kubernetes cluster (GKE).

GCP Cloud Build pipeline:

Build GCP infrastructure with Terraform code (VPC, GKE, Vault storage, etc.)

Deploy HashiCorp Vault with HELM chart

Configure HashiCorp Vault with Terraform code (auth, policies, mounts, etc.)

Test HashiCorp Vault GKE failover and RAFT replication

Test HashiCorp Vault configuration (app_roles, transit, etc.)

GitHub

Developer Workstation implementation in Azure cloud VMs

The project solves the problem "It works on my workstation..." by creating identical development environments in Azure cloud VMs for all developers working on the application.

Implemented features:

Network & VM deployment in Azure cloud (AZ CLI)

Docker-compose for application stack

MSSQL local & Azure Blob Storage backup & restore

Local secrets management

GitHub