<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Shrijal Acharya]]></title><description><![CDATA[Shrijal Acharya]]></description><link>https://shricodev.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Tue, 17 Nov 2026 03:21:26 GMT</lastBuildDate><atom:link href="https://shricodev.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[How I Manage my System and Dotfiles (with Ansible) 🔥]]></title><description><![CDATA[Wait... before I share the problem with you, I first need to make sure that you're on the right machine. I assume you are on Linux or macOS, as I am not so…]]></description><link>https://shricodev.hashnode.dev/how-i-manage-my-system-and-dotfiles-with-ansible</link><guid isPermaLink="true">https://shricodev.hashnode.dev/how-i-manage-my-system-and-dotfiles-with-ansible</guid><category><![CDATA[ansible]]></category><category><![CDATA[dotfiles]]></category><category><![CDATA[Linux]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Thu, 01 May 2025 12:10:21 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1746101346497/458b07e4-e154-4f09-83d7-52ef92845320.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-the-problem">The Problem 🤕</h2>
<p>Wait... before I share the problem with you, I first need to make sure that you're on the right machine. I assume you are on Linux or macOS, as I am not so sure if this plays well with Windows.</p>
<p>So, what's the problem we're trying to solve here?</p>
<p>As the title should already suggest, the idea is to configure and manage your system and dotfiles with <strong>Ansible</strong> (yes, you heard it correctly) and not GNU Stow.</p>
<p><img src="https://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExYnc5a3BrMXBpdmR2aTlkMDZueTNnYm1uMzJjcmRvYWhzM2YyZG5qNiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/95jaWzP8R4lSE/giphy.gif" alt="Shocked Dog" /></p>
<p>But why go through all this, right? Well, setting up a system is usually a one-time thing and not something you have to do over and over, when you mostly stick with one system.</p>
<p>But wait... what if you suddenly need to? Like, you've got two different machines, maybe one macOS and one Linux, or even two of the same kind, or you need to set up your office computer just the way you like it.</p>
<p>Now, do you see the pain that you <strong>might</strong> have to go through?</p>
<hr />
<h2 id="heading-the-solution">The Solution</h2>
<p>At the very least, managing your dotfiles is super important. If you are not already managing them properly, you're doing things really wrong, my friend.</p>
<p>There's two different efficient ways of handling this:</p>
<h3 id="heading-1-gnu-stow">1. GNU Stow</h3>
<p>With stow, you're mainly talking about "managing <s>system and</s> dotfiles," system management is completely out of play.</p>
<p>Most folks don't really care about automating system management and just prefer managing dotfiles, and that's perfectly fine. You'll find tons of different videos on YT explaining how to do it.</p>
<p>Here's the video that sold Stow to me a year ago: <a target="_blank" href="https://youtu.be/y6XCebnB9gs?si=2nH5j6Kez82HCVwx">Link</a></p>
<p>I've been using this method for over a year to handle my dotfiles, and you can check out my "Stow" way of managing dotfiles here: <a target="_blank" href="https://github.com/shricodev/dotfiles/tree/old-stow">Link</a></p>
<p>But, this is not the focus for this blog, let's continue to Ansible...</p>
<h3 id="heading-2-ansible">2. Ansible</h3>
<p>When you think of Ansible, you probably think of a tool for managing systems. But guess what? You can totally use it to handle your dotfiles too.</p>
<p>So, how did I come up with the idea to use Ansible for managing my system and dotfiles?</p>
<p>It wouldn't be fair if I didn't mention <a target="_blank" href="https://www.youtube.com/@techdufus">TechDufus</a>. His setup really looked slick and mine configuration is highly based on his.</p>
<p>You can check out my repository here: <a target="_blank" href="https://github.com/shricodev/dotfiles">Link</a></p>
<p>Everything's managed by Ansible, so if you know the basics, it's pretty easy to follow.</p>
<p>Here's the setup in action where I set up two fresh Ubuntu machines with my configuration: <a target="_blank" href="https://youtu.be/wXHfggMFbS0">Link</a></p>
<hr />
<h2 id="heading-conclusion">Conclusion</h2>
<p>With this setup, you can not only manage your dotfiles but also set up a new machine just the way you like it, including all the packages you need, with just one command.</p>
<p>Once you get the hang of this setup, I bet you won't need anything else anytime soon, since it even takes care of system management, which was something GNU Stow couldn't do.</p>
<p>My configuration is still new, and I'm working on tweaking the workflow to use Ansible.</p>
<p>Feel free to drop any suggestions or let me know if I'm missing something! ✌️</p>
<p><img src="https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExOGE5eGFhN2d3MjhtcmRpZHowdHNlaWRqcmtjemR0Z3NrMThhNTNqNSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/13QpHNxXQDqh42TPEW/giphy.gif" alt="Bye Bye GIF" /></p>
]]></content:encoded></item><item><title><![CDATA[(Neo)Vim Makes You a 10x DEV, and I'm not Kidding]]></title><description><![CDATA[Discover how (Neo)Vim can enhance your development efficiency and productivity in a terminal-based workflow. Embrace the Vim mindset today!]]></description><link>https://shricodev.hashnode.dev/neovim-makes-you-a-10x-dev-and-im-not-kidding</link><guid isPermaLink="true">https://shricodev.hashnode.dev/neovim-makes-you-a-10x-dev-and-im-not-kidding</guid><category><![CDATA[Linux]]></category><category><![CDATA[vim]]></category><category><![CDATA[neovim]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Mon, 10 Feb 2025 14:29:41 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1739175691692/b931082c-0bc6-4609-9eaf-60927db9cc3b.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-woah-thats-a-bold-statement">WOAH, that's a bold statement 🤨</h2>
<p>Wait, I'll explain...</p>
<p>So you've clicked on the post, which likely means you are from other code editor families (VS Code, Cursor) and what not...</p>
<p>Now that I am a Neovim user, I simply cannot stand GUI IDEs. I can relate to this video by <a target="_blank" href="https://www.youtube.com/@nirlichtman">Nir Lichtman</a> more and more as I am in the VIM workflow.</p>
<p><a target="_blank" href="https://www.youtube.com/watch?v=UTtitAidw2k">https://www.youtube.com/watch?v=UTtitAidw2k</a></p>
<p>This is by no means to say that anyone who uses GUI editors is not an efficient developer. <strong>DON'T GET IT WRONG!</strong></p>
<p>First, let me share my story before sharing anything else.</p>
<h2 id="heading-my-vim-journey-starts">My VIM journey starts</h2>
<p>When starting to code, everyone suggests using VS Code, so I started using it and that was my preferred choice of editor as well.</p>
<p>But after using it for 4 whole years, I started to get sick of it. Everyone suggests thousands of different extensions for one single niche in programming.</p>
<p>And I fell into the trap of installing most of the recommended plugins which I didn't even use ever. It was just sitting there in because it was so-called <strong>recommended extensions</strong>, and also you feel a bit of guilt to remove <strong>recommended</strong> things, right? 🫤 so they were just there.</p>
<p>One day, out of nowhere, this video from <a target="_blank" href="https://www.youtube.com/@typecraft_dev">typecraft</a>, got recommended in my YouTube feed.</p>
<p><a target="_blank" href="https://www.youtube.com/watch?v=zHTeCSVAFNY&amp;list=PLsz00TDipIffreIaUNk64KxTIkQaGguqn">From 0 to IDE in NEOVIM from scratch</a></p>
<p>I was so tempted to check out what <strong>VIM</strong> really is. Before this, I never used VIM because once, when I entered VIM accidentally, I had to spend quite some time figuring out just how to close it 🫤, so didn't even have a thought of using it. So, for any terminal work that required using an editor, I used <strong>nano</strong>.</p>
<p>So, I finally decided to take a look into what all this hype for VIM in developers was really about.</p>
<p>To my surprise, I was really hooked on the VIM ecosystem in no time. Then I did all the <strong>LSP configuration</strong>, <strong>autoformatting</strong>, <strong>fuzzy finding</strong>, and more, which you get pre-built in other GUI IDEs. I loved manually configuring everything down to the minutest details to my preferences.</p>
<p>I got to know how all of this works behind the scenes, and I already knew VIM was my thing.</p>
<p>Then, it was time to learn some VIM motions, so I started learning VIM motions from YouTube. After watching quite a few videos on VIM motions, it didn’t really feel like I knew VIM motions and everything else in VIM more than the basics.</p>
<p>So, that led me to read this fantastic book <strong>"Practical Vim"</strong> by <strong>Drew Neil</strong>.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d4nvcmggs8mhu6w9oj6i.png" alt="Practical Vim Cover Page" /></p>
<p>This book introduced me to the soul of VIM. I’m never a book-reading guy, but I read this <strong>356-page</strong> book that just talks about VIM. 😴</p>
<p>From this, as you can imagine, I am already quite into the VIM ecosystem and have no thoughts of going back to any other IDEs, at least for now.</p>
<h2 id="heading-what-next">What Next?</h2>
<p>When I started learning VIM, I thought learning VIM was <strong>just learning VIM</strong>, but I was completely wrong!</p>
<p>It introduces you to the whole ecosystem of developer tools. After entering this terminal ecosystem of editors, I learned quite a few other developer tools like <strong>tmux</strong>, <strong>stow</strong>, window managers like <strong>i3</strong>, and overall, the importance of understanding things by reading and doing rather than watching some YouTube videos and learning.</p>
<p>Using VIM and entering this complete terminal ecosystem actually convinced me to completely switch from <strong>Windows with WSL2</strong> to a <strong>full-time Linux user with Arch</strong>.</p>
<p><img src="https://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExb2hhN3VsNDUwdmJtaDU0eXN5bTE5a3FndHF6bGV5MG0xbnR5YXRkNSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/rVVFWyTINqG7C/giphy.gif" alt="Mind Blown GIF" /></p>
<p>For me, it was <strong>Arch</strong>, for you, it might be something else, but you will find reasons for transitioning into Linux.</p>
<p>Along the way, I got to know some of the OGs like <a target="_blank" href="https://www.youtube.com/@ThePrimeagen">ThePrimeagen</a>, <a target="_blank" href="https://www.youtube.com/c/tjdevries">TjDevries</a> .... the Mario and Luigi of the Neovim cult.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l6c714redxyko7t0zir1.jpg" alt="Primeagen and Tj Devries" /></p>
<h2 id="heading-how-exactly-a-10x-dev">How exactly a 10x DEV?</h2>
<p>The phrase <strong>10x dev</strong> sounds a bit weird, but to me, a 10x dev is really just a super-efficient DEV. And let me tell you, VIM is like the cheat code to becoming one.</p>
<p>After using VIM for quite some time now, it feels great to say that I don’t even have a GUI editor installed on my machine. It’s just me, my terminal, and VIM. And honestly, I’ve never been more productive.</p>
<p>Here’s the thing: VIM forces you to think differently. It’s not just about typing code, it’s about <strong>editing text efficiently</strong>. Once you get the hang of VIM motions, you start to realize how much time you’ve been wasting with your mouse or arrow keys. You start to think in terms of <strong>actions</strong> and <strong>movements</strong>. Want to delete a word? <code>dw</code>. Delete a line? <code>dd</code>. Need to jump to the end of the file? <code>G</code>. It’s that simple.</p>
<p>And the best part? VIM is everywhere. Once you’re comfortable with it, you can <strong>SSH</strong> into any server, open any file, and feel right at home. No need to install a bulky IDE or configure a million plugins. It’s just VIM, and it’s always there.</p>
<p><img src="https://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExZTgwa3M0d2cxYzdpbnljcmhqem15Y250M2Vma2k2YmgwaDduMzhobCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/Ws6T5PN7wHv3cY8xy8/giphy.gif" alt="Typing GIF" /></p>
<p>So, is VIM the secret to becoming a 10x dev? <strong>Maybe</strong>. But more importantly, it’s about adopting a mindset of efficiency. VIM is just the tool that helps you get there.</p>
<p>And hey, if nothing else, at least you can say <strong>I use VIM, BTW! 😂</strong></p>
<hr />
<h2 id="heading-wrap">Wrap!</h2>
<p>So, there you have it! This is my journey from being a GUI-loving dev who cared more about the aesthetics of the editor to becoming a terminal-obsessed VIM fellow.</p>
<p>If you are still on the thoughts of giving VIM a try? I say <strong>go for it.</strong></p>
<p>You will definitely feel the power that comes with staying in the terminal overall.</p>
]]></content:encoded></item><item><title><![CDATA[Be a 10x AI Developer with these 5 tools in 2025 ✅ 🚀]]></title><description><![CDATA[Discover top 5 AI tools for developers in 2025, including pipeline integration, image generation, AI copilots, and web app builders. 🚀]]></description><link>https://shricodev.hashnode.dev/be-a-10x-ai-developer-with-these-5-tools-in-2025</link><guid isPermaLink="true">https://shricodev.hashnode.dev/be-a-10x-ai-developer-with-these-5-tools-in-2025</guid><category><![CDATA[AI]]></category><category><![CDATA[Machine Learning]]></category><category><![CDATA[Programming Blogs]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Mon, 27 Jan 2025 15:33:06 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1737991854456/af7ec654-bcc9-49d5-9441-d7827e1208dd.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR</h2>
<p>This article lists my top 5 AI tools I use daily to be a 10x AI developer in 2025. ✅</p>
<p>These tools are mostly aimed at <strong>integrating missing AI pipelines</strong>, <strong>image generation</strong>, <strong>plug-and-play AI copilots</strong>, and <strong>AI web app builders</strong>. 🔥</p>
<blockquote>
<p>If you are in the development and AI field, I can tell you'll also be using at least a few in your daily life. 🤫</p>
</blockquote>
<p><img src="https://media0.giphy.com/media/v1.Y2lkPTc5MGI3NjExN3Q3ZmF0cG94MDVhdnVrdDNsMmZkaTNpZW9ub3FjbnJhMjUwM3lpeiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/CLQNFY6YIHanjWSEhW/giphy.gif" alt="Shocked Orange GIF" /></p>
<hr />
<h2 id="heading-1-kitopshttpskitopsml-ai-pipeline-integration">1. <a target="_blank" href="https://kitops.ml">KitOps</a> <strong>- AI pipeline integration</strong></h2>
<blockquote>
<p>ℹ️ Open-source DevOps tool for packaging and versioning AI/ML projects.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qyrbegkc36nmyyqtzstj.png" alt="KitOps ML" /></p>
<p>KitOps is an open-source DevOps standard packaging and versioning tool that does simple and secure packaging of AI/ML projects into reproducible artifact called a ModelKit.</p>
<p>ModelKits represent a complete bundle of a project's artifacts, including models, datasets, and code.</p>
<p>ModelKits can be stored in a container registry, providing a way to track and audit the history of changes. ModelKits are immutable and work with all existing tools, allowing you to use the same deployment pipelines and endpoints as in your development process.</p>
<blockquote>
<p>💡 <strong>NOTE</strong>: A ModelKit can contain only one model but can include multiple datasets or codebases.</p>
</blockquote>
<h3 id="heading-problem-with-the-traditional-workflow">😮‍💨 <strong>Problem with the Traditional Workflow</strong></h3>
<p>The traditional workflow involves moving a model from Jupyter Notebook to an ML tool, then manually deploying it to a working dev/production server like Kubernetes, which is difficult and slow.</p>
<h3 id="heading-this-is-where-kitops-shines">😎 This is where KitOps shines!</h3>
<p>KitOps is built to standardize the entire traditional process of packaging, deployment, and tracking of AI/ML models, making them universal and capable of running anywhere, similar to application code.</p>
<p><strong>Introducing Kitfile</strong></p>
<p>At the heart of any AI or ML project managed by KitOps is a <strong>Kitfile</strong>. It's a <strong>YAML-based manifest file</strong> that helps share the project artifacts. The Kitfile is stored with the ModelKit.</p>
<p>Think of it as the blueprint of your project, ensuring that all the components in your application are properly organized and accessible.</p>
<p>You can extract a Kitfile from a ModelKit in the current directory with a single command:</p>
<pre><code class="lang-bash">kit unpack [registry/repo:tag] --config -d .
</code></pre>
<p>There are 4 main section to a Kitfile,</p>
<ul>
<li><p><code>package</code>: Stores the ModelKit metadate like author details, name, description.</p>
</li>
<li><p><code>code</code>: Path to the Jupyter Notebook directory.</p>
</li>
<li><p><code>model</code>: Path to the serialized Model.</p>
</li>
<li><p><code>datasets</code>: Path to the datasets.</p>
</li>
</ul>
<p>A Kitfile may or may not have all of the sections. The only mandatory compulsion is that it needs to have the <code>manifestVersion</code> and at least one of <code>code</code>, <code>model</code>, <code>docs</code> or <code>datasets</code>.</p>
<p>Here's how a sample Kitfile looks which introduced a pair of datasets:</p>
<pre><code class="lang-yaml"><span class="hljs-attr">manifestVersion:</span> <span class="hljs-string">v1.0.0</span>

<span class="hljs-attr">datasets:</span>
<span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">training</span> <span class="hljs-string">data</span>
  <span class="hljs-attr">path:</span> <span class="hljs-string">./data/train.csv</span>
<span class="hljs-bullet">-</span> <span class="hljs-attr">description:</span> <span class="hljs-string">validation</span> <span class="hljs-string">data</span> <span class="hljs-string">(tabular)</span>
  <span class="hljs-attr">name:</span> <span class="hljs-string">validation</span> <span class="hljs-string">data</span>
  <span class="hljs-attr">path:</span> <span class="hljs-string">./data/test.csv</span>
</code></pre>
<blockquote>
<p>⚠️ <strong>NOTE</strong>: You must always use relative paths in a Kitfile. Absolute paths are not allowed.</p>
</blockquote>
<p>This ModelKit and Kitfile workflow not only standardizes the steps but also minimizes the risk of mistakes that can occur in manual processes.</p>
<h3 id="heading-standard-kitops-workflow">Standard KitOps Workflow</h3>
<blockquote>
<p>ℹ️ These are the basic steps involved in the KitOps workflow.</p>
</blockquote>
<ul>
<li><strong>Installation</strong> 🛠️</li>
</ul>
<p>To work with KitOps, first, you need to install it locally, follow the installation guide available on their page <a target="_blank" href="https://kitops.ml/docs/cli/installation">here</a>.</p>
<ul>
<li><strong>Initialize Kitfile</strong> 🗺️</li>
</ul>
<p>To create a Kitfile, run the following command:</p>
<pre><code class="lang-bash">kit init .
</code></pre>
<p>This command creates a <strong>Kitfile</strong> in your current directory. It will define dependencies, models, datasets, and configurations.</p>
<ul>
<li><strong>Pack and Push the ModelKit to registry</strong> 🚀</li>
</ul>
<p>Suppose, you completed fine-tuning of your model and the registry of your choice is <strong>GitLab</strong>(<code>registry.gitlab.com</code>), the repository is called <code>chat-assitant</code> and the name of the model is <code>aichat</code>, then you can use the following command to pack and push it to the registry:</p>
<pre><code class="lang-bash">kit pack . -t registry.gitlab.com/chat-assistant/aichat:tuned
kit push registry.gitlab.com/chat-assistant/aichat:tuned
</code></pre>
<p>Now, as you push the changes, all the members in your team can be notified of the change in the model and can pull, run and inspect the model.</p>
<p>Now, once the model completes the training or the tuning, the team can then pull the updated model with the <code>kit pull</code> command:</p>
<pre><code class="lang-bash">kit pull registry.gitlab.com/chat-assistant/aichat:tuned
</code></pre>
<h3 id="heading-hugging-face-import-workflow">Hugging Face Import Workflow 🕵️‍♂️</h3>
<p>After the recent release of <a target="_blank" href="https://jozu.com/blog/kitops-v1-0-0-release-announcement/">KitOps v1.0.0</a>, you can use <code>kit import</code> command to import any <strong>HuggingFace model</strong> and convert it into a ModelKit that you can push to image registries such as DockerHub.</p>
<p>💁 <strong>When you run <code>microsoft/phi-4</code>, the Kit CLI will:</strong></p>
<ol>
<li><p>Download the <code>microsoft/phi-4</code> huggingface repository locally.</p>
</li>
<li><p>Generate the configuration required to package that repository into a ModelKit.</p>
</li>
<li><p>Package the repository into a locally-stored ModelKit</p>
</li>
</ol>
<p>Once this is done, you can simply run the following command to push  it to the registry and share it with the team. 😮</p>
<pre><code class="lang-bash">kit push microsoft/phi-4:latest docker.io/my-organization/phi-4:latest
</code></pre>
<p>As you can already imagine, using <strong>KitOps</strong> saves a significant amount of development time for a team. With everything stored in one place, it becomes incredibly convenient to query and work with.</p>
<p>It also makes it easy to trace which models are running, where they were pulled from, and the status of the running models. 😎</p>
<hr />
<h2 id="heading-2-nebius-ai-studiohttpsnebiuscom-ai-models-like-image-generation">2. <a target="_blank" href="https://nebius.com/">Nebius AI Studio</a> <strong>- AI models like image-generation</strong></h2>
<blockquote>
<p>ℹ️ The Nebius AI Studio is not just limited to image-generation, it's a suite of AI models. You can find all <a target="_blank" href="https://docs.nebius.com/studio/inference/api#models">here</a>.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/sggfd1hbd4klk590pw4p.png" alt="Nebius AI Studio" /></p>
<p>There are tons of AI models available in Nebius to fine-tune your image generation.</p>
<p>And the best part is they are super affordable and already come with <strong>$1 credit</strong> to try out the models before purchasing.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zuqaidqknf9meneh0ukb.png" alt="Nebius AI Studio models" /></p>
<p>Recently, my friend @arindam_1729 built a project that uses the Nebius models, like <strong>Flux Schnell</strong> and <strong>SDXL 1.0</strong>, to generate logos based on user prompts.</p>
<blockquote>
<p>💡 If you wish to give it a look, check out this <a target="_blank" href="https://github.com/Arindam200/logo-ai">repository</a>. </p>
</blockquote>
<p>To add text-to-image generation to your application, it is as simple as adding the following lines of code:</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">from</span> openai <span class="hljs-keyword">import</span> OpenAI
client = OpenAI()

client.images.generate(
    model=<span class="hljs-string">"stability-ai/sdxl"</span>,
    prompt=<span class="hljs-string">"An elephant in a desert"</span>,
    response_format=<span class="hljs-string">"b64_json"</span>,
    extra_body={
        <span class="hljs-string">"response_extension"</span>: <span class="hljs-string">"webp"</span>,
        <span class="hljs-string">"width"</span>: <span class="hljs-number">512</span>,
        <span class="hljs-string">"height"</span>: <span class="hljs-number">512</span>,
        <span class="hljs-string">"num_inference_steps"</span>: <span class="hljs-number">30</span>,
        <span class="hljs-string">"seed"</span>: <span class="hljs-number">-1</span>,
        <span class="hljs-string">"negative_prompt"</span>: <span class="hljs-string">"Giraffes, night sky"</span>
    }
)
</code></pre>
<p>You can also interact with Nebius Cloud through your command line. 🧑‍💻</p>
<p>To get started with Nebius, simply fire up the following <code>curl</code> command:</p>
<pre><code class="lang-bash">curl -sSL https://storage.eu-north1.nebius.cloud/cli/install.sh | bash
</code></pre>
<p>Re-source your shell with <code>exec -l $SHELL</code>, and to check if you have successfully installed it, try running <code>nebius version</code>.</p>
<hr />
<h2 id="heading-3-copilotkithttpswwwcopilotkitai-seamless-ai-copilots-ready-to-go">3. <a target="_blank" href="https://www.copilotkit.ai/">CopilotKit</a> <strong>- Seamless AI Copilots, Ready to Go</strong></h2>
<blockquote>
<p>ℹ️ Virtual colleague into your product that fully understands your application &amp; your users.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6dxaoxdp3qu8vi5z9a1j.png" alt="CopilotKit" /></p>
<p>CopilotKit is not just another AI application with chatbot features, it can understand the real-time state of your application and make changes as needed.</p>
<blockquote>
<p>I am a big user of this tool, as I have built a couple of projects using CopilotKit. You can find them all on my <a target="_blank" href="https://github.com/shricodev?tab=repositories">GitHub profile</a>.</p>
<p>🤔 <strong>Why use CopilotKit as compared to other alternatives?</strong></p>
</blockquote>
<p>If you are not convinced why you should bother trying out yet another AI addition to your application, let me show you some reasons to consider using CopilotKit.</p>
<p>✅ <strong>Super Easy to integrate</strong></p>
<p>To add CopilotKit in any application, it is just as simple as wrapping your application in the <code>&lt;CopilotKit /&gt;</code> provider.</p>
<pre><code class="lang-jsx"><span class="hljs-keyword">import</span> <span class="hljs-string">"./globals.css"</span>;

<span class="hljs-keyword">import</span> { ReactNode } <span class="hljs-keyword">from</span> <span class="hljs-string">"react"</span>;
<span class="hljs-keyword">import</span> { CopilotKit } <span class="hljs-keyword">from</span> <span class="hljs-string">"@copilotkit/react-core"</span>; 

<span class="hljs-keyword">export</span> <span class="hljs-keyword">default</span> <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">RootLayout</span>(<span class="hljs-params">{ children }: { children: ReactNode }</span>) </span>{
    <span class="hljs-keyword">return</span> (
      <span class="xml"><span class="hljs-tag">&lt;<span class="hljs-name">html</span> <span class="hljs-attr">lang</span>=<span class="hljs-string">"en"</span>&gt;</span>
        <span class="hljs-tag">&lt;<span class="hljs-name">body</span>&gt;</span> 
          {/* Use the public api key you got from Copilot Cloud  */}
          <span class="hljs-tag">&lt;<span class="hljs-name">CopilotKit</span> <span class="hljs-attr">publicApiKey</span>=<span class="hljs-string">"&lt;your-copilot-cloud-public-api-key&gt;"</span>&gt;</span> 
            {children}
          <span class="hljs-tag">&lt;/<span class="hljs-name">CopilotKit</span>&gt;</span>
        <span class="hljs-tag">&lt;/<span class="hljs-name">body</span>&gt;</span>
      <span class="hljs-tag">&lt;/<span class="hljs-name">html</span>&gt;</span>
    );
}</span>
</code></pre>
<p>✅ <strong>Comes with Built-in UI</strong></p>
<blockquote>
<p>ℹ️ Copilot UI ships with a number of built-in UI patterns, choose whichever one you like.</p>
</blockquote>
<p>All the UI components are pre-built for you to plug in and get started, with no need to design anything from scratch.</p>
<p>You can pick from <code>CopilotChat</code>, <code>CopilotPopup</code>, and <code>CopilotSidebar</code>.</p>
<p>To add any UI component, it’s as simple as importing the component and placing it in the desired location:</p>
<pre><code class="lang-jsx"><span class="hljs-keyword">import</span> { CopilotPopup } <span class="hljs-keyword">from</span> <span class="hljs-string">"@copilotkit/react-ui"</span>;

<span class="hljs-keyword">export</span> <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">YourApp</span>(<span class="hljs-params"></span>) </span>{
  <span class="hljs-keyword">return</span> (
    <span class="xml"><span class="hljs-tag">&lt;&gt;</span>
      <span class="hljs-tag">&lt;<span class="hljs-name">YourMainContent</span> /&gt;</span>
      <span class="hljs-tag">&lt;<span class="hljs-name">CopilotPopup</span>
        <span class="hljs-attr">instructions</span>=<span class="hljs-string">{</span>"<span class="hljs-attr">You</span> <span class="hljs-attr">are</span> <span class="hljs-attr">assisting</span> <span class="hljs-attr">the</span> <span class="hljs-attr">user</span> <span class="hljs-attr">as</span> <span class="hljs-attr">best</span> <span class="hljs-attr">as</span> <span class="hljs-attr">you</span> <span class="hljs-attr">can.</span> <span class="hljs-attr">Answer</span> <span class="hljs-attr">in</span> <span class="hljs-attr">the</span> <span class="hljs-attr">best</span> <span class="hljs-attr">way</span> <span class="hljs-attr">possible</span> <span class="hljs-attr">given</span> <span class="hljs-attr">the</span> <span class="hljs-attr">data</span> <span class="hljs-attr">you</span> <span class="hljs-attr">have.</span>"}
        <span class="hljs-attr">labels</span>=<span class="hljs-string">{{</span>
          <span class="hljs-attr">title:</span> "<span class="hljs-attr">Popup</span> <span class="hljs-attr">Assistant</span>",
          <span class="hljs-attr">initial:</span> "<span class="hljs-attr">Need</span> <span class="hljs-attr">any</span> <span class="hljs-attr">help</span>?",
        }}
      /&gt;</span>
    <span class="hljs-tag">&lt;/&gt;</span></span>
  );
}
</code></pre>
<p>The steps will be very similar for all the other available components. You can always refer to the link above to learn more.</p>
<p>If you want to design everything on your own, there is also an option for Headless UI, where you can design everything to suit your needs. 🔥</p>
<p>✅ <strong>CoAgents Support</strong></p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mo0nh8fxyrhrythf1knd.png" alt="CopilotKit Coagents" /></p>
<p>If you want to use your custom agents, you can do so as well. With this, you can have a shared state between your agent and the application, an agentic generative UI, add a human-in-the-loop stage, and implement real-time frontend actions using the <code>useCoagentStateRender()</code> and <code>useCoAgent()</code> hooks.</p>
<p>There's an example travel application they have to demonstrate the use of custom agents, check out <a target="_blank" href="https://docs.copilotkit.ai/coagents/tutorials/ai-travel-app#ai-travel-agentic-copilot-tutorial">here</a>.</p>
<hr />
<h2 id="heading-4-latitudehttpslatitudeso-build-test-and-deploy-llm-features">4. <a target="_blank" href="https://latitude.so">Latitude</a> <strong>- Build, Test and Deploy LLM features</strong></h2>
<blockquote>
<p>ℹ️ Latitude helps you refine your prompts with data to deliver reliable AI products with confidence.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v0acqvlwqhkaiulqvnv3.png" alt="Latitude - Build, Test, Deploy LLM features" /></p>
<p>Latitude is a platform that helps developers and product teams build AI features with confidence. Track and evaluate your prompts, improve them using real data, and deploy new changes easily.</p>
<blockquote>
<p>The best part about Latitude is that it gives you 40K prompt and evaluation runs every month for free. 🤯</p>
</blockquote>
<p>It’s designed to be easy to use, so you can get started quickly and iterate on your prompts without needing to worry about the infrastructure or tooling.</p>
<blockquote>
<p>💡 Long story short, it is a prompt-engineering platform that helps you build or iterate on your prompts before shipping it to the production so the output from the LLM is always what you would expect.</p>
</blockquote>
<p>There's two-way of getting started with Latitude:</p>
<ul>
<li><p><strong>Latitude Cloud</strong>: A fully managed solution that allows you to get started quickly without worrying about infrastructure.</p>
</li>
<li><p><strong>Latitude Self-Hosted</strong>: An open-source version that you can deploy and manage on your own infrastructure for complete control and customization.</p>
</li>
</ul>
<hr />
<h2 id="heading-5-taipyhttpstaipyio-python-data-and-bi-webapp-builder">5. <a target="_blank" href="https://taipy.io">Taipy</a> <strong>- Python Data and BI Webapp Builder</strong></h2>
<blockquote>
<p>ℹ️ From simple pilots to production-ready web applications in no time.</p>
</blockquote>
<p>With Taipy you can turn your Data and AI algorithms into production-ready web applications.</p>
<blockquote>
<p>🤔 <strong>Who is Taipy for?</strong></p>
</blockquote>
<p>Taipy is designed for data scientists and machine learning engineers to build data &amp; web applications.</p>
<h3 id="heading-key-features-of-taipy">Key Features of Taipy:</h3>
<ul>
<li>✅ Seamlessly embed ML models and data science workflows</li>
<li>✅ Easy to deploy anywhere in your custom domain</li>
<li>✅ For a VS Code user, Taipy has an extension that unlocks a graphical editor</li>
<li>✅ Taipy enhances performance with caching control of graphical events</li>
</ul>
<p>To get started with Taipy, first, make sure that you have Taipy installed:</p>
<pre><code class="lang-bash">pip install taipy
</code></pre>
<p>Creating a simple sample dynamic web application in Taipy can be easily done with a few lines of Python code.</p>
<p>Create a Python file and paste the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-keyword">from</span> taipy.gui <span class="hljs-keyword">import</span> Gui
<span class="hljs-keyword">import</span> taipy.gui.builder <span class="hljs-keyword">as</span> tgb
<span class="hljs-keyword">from</span> math <span class="hljs-keyword">import</span> cos, exp

value = <span class="hljs-number">10</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">compute_data</span>(<span class="hljs-params">decay:int</span>)-&gt;list:</span>
    <span class="hljs-keyword">return</span> [cos(i/<span class="hljs-number">6</span>) * exp(-i*decay/<span class="hljs-number">600</span>) <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(<span class="hljs-number">100</span>)]

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">slider_moved</span>(<span class="hljs-params">state</span>):</span>
    state.data = compute_data(state.value)

<span class="hljs-keyword">with</span> tgb.Page() <span class="hljs-keyword">as</span> page:
    tgb.text(value=<span class="hljs-string">"# Taipy Getting Started"</span>, mode=<span class="hljs-string">"md"</span>)
    tgb.text(value=<span class="hljs-string">"Value: {value}"</span>)
    tgb.slider(value=<span class="hljs-string">"{value}"</span>, on_change=slider_moved)
    tgb.chart(data=<span class="hljs-string">"{data}"</span>)

data = compute_data(value)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    Gui(page=page).run(title=<span class="hljs-string">"Dynamic chart"</span>)
</code></pre>
<p>Just by adding this code, we have a graph and a slider component which helps adjust the data parameter.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4sg4ljg9moncrdg1zkdh.png" alt="Taipy Data Visualization" /></p>
<p>There are many others like bar chart visualization. To learn more, visit <a target="_blank" href="https://taipy.io">Taipy</a>.</p>
<blockquote>
<p>💡 Taipy simplifies the process of turning complex Python algorithms into user-friendly web applications with just a few lines of Python, without requiring knowledge of additional languages.</p>
</blockquote>
<hr />
<blockquote>
<p>If you think of any other handy AI tools that I haven't covered in this article, do share them in the comments section below. 👇🏻</p>
</blockquote>
<p>So, that is it for this article. Thank you so much for reading! 🎉🫡</p>
<p><img src="https://media0.giphy.com/media/v1.Y2lkPTc5MGI3NjExNHNzcTJiNWl1a2w3aHRvdjhkdGtxdHViZGxpYXZtM2tpYTdsb2tnMyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/3o6EhGvKschtbrRjX2/giphy.gif" alt="Bye Bye Ryan Gosling GIF" /></p>
]]></content:encoded></item><item><title><![CDATA[🤖 Build an AI Travel Planner with CopilotKit, LangGraph & Google Maps API 🤩]]></title><description><![CDATA[Build an AI travel planner using CopilotKit, LangGraph, and Google Maps API. Enhance your app with AI for real-time updates and automated workflows]]></description><link>https://shricodev.hashnode.dev/build-an-ai-travel-planner-with-copilotkit-langgraph-google-maps-api</link><guid isPermaLink="true">https://shricodev.hashnode.dev/build-an-ai-travel-planner-with-copilotkit-langgraph-google-maps-api</guid><category><![CDATA[langgraph]]></category><category><![CDATA[AI]]></category><category><![CDATA[google map]]></category><category><![CDATA[Next.js]]></category><category><![CDATA[React]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Sat, 25 Jan 2025 09:28:48 GMT</pubDate><content:encoded><![CDATA[<p>💡 <strong>NOTE</strong>: This article was ghostwritten by me and originally published under CopilotKit on <a target="_blank" href="https://dev.to/copilotkit/build-an-ai-travel-planner-with-copilotkit-langgraph-google-maps-api-32fm">DEV</a> under the name <a target="_blank" href="https://dev.to/arindam_1729">Arindam Majumder</a> for <a target="_blank" href="https://dev.to/studio1hq">Studio1</a> on Jan 16, 2025.</p>
<h2 id="heading-tldr">TL;DR</h2>
<p>In this easy-to-follow tutorial, we’ll take a simple travel planner application and enhance it with AI using <strong>CopilotKit</strong>.</p>
<p><strong>By the end of this article, you’ll learn:</strong></p>
<ul>
<li><p>What an agentic copilot is and how to use it to add AI to your application.</p>
</li>
<li><p>How to let the copilot update the app state and render changes in real time.</p>
</li>
<li><p>How to use <code>useCoAgentStateRender</code> for human-in-the-loop workflows.</p>
</li>
</ul>
<blockquote>
<p>💁 Prefer learning through videos? Check out <a target="_blank" href="https://youtu.be/9v3kXiOY3vg?si=b7hBDCMx5Xu44MIj">this video tutorial</a>.</p>
<p>💁 Interact with the travel planner demo by visiting <a target="_blank" href="https://examples-coagents-ai-travel-app.vercel.app/">this link</a>.</p>
</blockquote>
<hr />
<h2 id="heading-check-out-the-repository">Check out the repository</h2>
<blockquote>
<p>For this demonstration, we’ll start with a branch containing a basic, functional application without AI support, which we’ll enhance using CopilotKit. 🚀</p>
</blockquote>
<h3 id="heading-check-out-the-starting-branch">Check out the Starting Branch</h3>
<p>We’ll begin with the <code>coagents-travel-tutorial-start</code> branch, which contains the starter code for our travel app:</p>
<pre><code class="lang-bash">git <span class="hljs-built_in">clone</span> -b coagents-travel-tutorial-start https://github.com/CopilotKit/CopilotKit.git
<span class="hljs-built_in">cd</span> CopilotKit
</code></pre>
<p>The tutorial code is located in the <code>examples/coagents-travel</code> directory, which includes two different directories:</p>
<ul>
<li><p><code>ui/</code>: It holds a Next.js application where we’ll integrate the LangGraph agent.</p>
</li>
<li><p><code>agent/</code>: It holds a Python-based LangGraph agent.</p>
</li>
</ul>
<p>Navigate to the <code>examples/coagents-travel</code> directory:</p>
<pre><code class="lang-bash"><span class="hljs-built_in">cd</span> examples/coagents-travel
</code></pre>
<h3 id="heading-install-dependencies">Install Dependencies</h3>
<p>Let’s set up the Next.js application. Ensure you have <code>pnpm</code> installed on your system, as our starter code uses it as the package manager:</p>
<pre><code class="lang-bash">npm install -g pnpm@latest-10
</code></pre>
<p>Now, go to the <code>ui</code> directory and install all the required dependencies for the project:</p>
<pre><code class="lang-bash"><span class="hljs-built_in">cd</span> ui
pnpm install
</code></pre>
<h3 id="heading-retrieve-api-keys">Retrieve API Keys</h3>
<p>Create a <code>.env</code> file in the <code>ui</code> directory and populate it with the necessary environment variables:</p>
<pre><code class="lang-plaintext"># 👇 ui/.env

OPENAI_API_KEY=&lt;your_openai_api_key&gt;
NEXT_PUBLIC_CPK_PUBLIC_API_KEY=&lt;your_public_copilotkit_api_key&gt;
</code></pre>
<p>If you need a CopilotKit API key, you can grab one <a target="_blank" href="https://cloud.copilotkit.ai">here</a>. 🔑</p>
<h3 id="heading-start-the-project">Start the Project</h3>
<p>Now, that we have all the dependencies installed, launch the development server:</p>
<pre><code class="lang-bash">pnpm run dev
</code></pre>
<p>If everything is set up correctly, visit <a target="_blank" href="http://localhost:3000">http://localhost:3000</a> to see the travel app in action. 😻</p>
<p>Now, we’ll take a look into the LangGraph agent and see how it works.</p>
<hr />
<h2 id="heading-langgraph-agent">LangGraph Agent</h2>
<p>Before we dive into integrating the LangGraph agent, let’s take a moment to understand how it works.</p>
<p>For this tutorial, we won’t be building the LangGraph agent from scratch. Instead, we’ll use the prebuilt version located in the <code>agent</code> directory.</p>
<blockquote>
<p>💁 Interested in a detailed, step-by-step guide on building a LangGraph agent? Check out the <a target="_blank" href="https://langchain-ai.github.io/langgraph/tutorials/introduction">LangGraph Quickstart Guide</a>.</p>
</blockquote>
<p>Let’s walk through the LangGraph agent to understand its inner workings before adding to our application.</p>
<h3 id="heading-install-langgraph-studio">Install LangGraph Studio</h3>
<blockquote>
<p>💡 LangGraph Studio is an excellent tool for visualizing and debugging LangGraph workflows. While not required for using CopilotKit, it’s highly recommended for understanding how LangGraph operates.</p>
</blockquote>
<p>To install LangGraph Studio, refer to the <a target="_blank" href="https://studio.langchain.com">LangChain Studio Setup Guide</a>.</p>
<h3 id="heading-retrieve-api-keys-1">Retrieve API Keys</h3>
<p>Create a <code>.env</code> file in the <code>agent</code> directory and populate it with the following environment variables:</p>
<pre><code class="lang-plaintext"># 👇 agent/.env

OPENAI_API_KEY=&lt;your_openai_api_key&gt;
GOOGLE_MAPS_API_KEY=&lt;your_google_maps_api_key&gt;
</code></pre>
<p>Need a Google Maps API key? Follow this <a target="_blank" href="https://developers.google.com/maps/documentation/javascript/get-api-key">guide</a> to get one. 🔑</p>
<h3 id="heading-visualizing-the-langgraph-agent">Visualizing the LangGraph Agent</h3>
<p>With LangGraph Studio installed, open the <code>examples/coagents-travel/agent</code> directory in the studio to load and visualize the LangGraph agent.</p>
<blockquote>
<p>💡 <strong>Tip:</strong> It might take a little time to get everything set up, but once it’s installed and you've navigated to the agent folder, the visualization will look something like this:</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a1t999iycfbbm2cowyqq.png" alt="LangGraph Visualization" /></p>
<h3 id="heading-testing-the-langgraph-agent">Testing the LangGraph Agent</h3>
<p>To test the LangGraph agent, simply add a message to the <code>messages</code> state variable and hit "Submit."</p>
<p>The agent will process the input, respond in the chat, and follow the defined workflow through the connected nodes.</p>
<p>In this example, the agent triggers the <code>search_node</code> to perform a search. Once it retrieves a response, it uses the <code>trips_node</code> to update the state by adding a new trip based on its findings. 🎯</p>
<h3 id="heading-understanding-breakpoints">Understanding Breakpoints</h3>
<p>Let’s talk about a key concept in agentic copilots: <a target="_blank" href="https://docs.copilotkit.ai/coagents/concepts/human-in-the-loop">human-in-the-loop</a>.</p>
<p>Imagine your agent is eager to help but sometimes just a little too eager. Breakpoints are like a friendly pause button, letting users step in and approve the agent’s decisions before it goes rogue (or just makes a mistake). LangGraph makes this easy with <strong>breakpoints</strong>.</p>
<p>Here’s how it works:</p>
<ul>
<li><p>Click on the <code>trips_node</code> and enable the <code>interrupt_after</code> option.</p>
</li>
<li><p>Try having the agent create a new trip. This time, it’ll stop mid-action and ask for your approval.</p>
</li>
</ul>
<p>See? Even AI can learn good manners. 🙃</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ez34gss876c0tlpbtwzr.png" alt="LangGraph Studio progress" /></p>
<h3 id="heading-leave-langgraph-studio-running">Leave LangGraph Studio running</h3>
<p>Your agent needs a home, and for now, LangGraph Studio is where it’s staying at. Leave the studio running locally; you’ll see its URL in the bottom-left corner of the application.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x45hmt3k8mcw0no3stea.png" alt="LangGraph Studio URL" /></p>
<p>We’ll use this URL later to connect our CopilotKit setup to the LangGraph agent.</p>
<p>We've done a great job so far, Now let's put all of this to work by integrating the LangGraph agent into our travel app as an agentic copilot. 🤖</p>
<hr />
<h2 id="heading-setting-up-copilotkit">Setting Up CopilotKit</h2>
<p>Now for the fun part — let's add CopilotKit to bring everything together. Since we have both the application and the agent running, we're just one step away from integrating CopilotKit into our application.</p>
<p>For this tutorial, we will install the following dependencies:</p>
<ul>
<li><p><code>@copilotkit/react-core</code>: The core library for CopilotKit, which contains the CopilotKit provider and useful hooks.</p>
</li>
<li><p><code>@copilotkit/react-ui</code>: The UI library for CopilotKit, which contains the CopilotKit UI components such as the sidebar, chat popup, textarea, and more.</p>
</li>
</ul>
<h3 id="heading-install-dependencies-1">Install Dependencies</h3>
<p>First, navigate to the <code>ui</code> directory if you’re not already there:</p>
<pre><code class="lang-bash"><span class="hljs-built_in">cd</span> ../ui
</code></pre>
<p>Then, install the CopilotKit packages:</p>
<pre><code class="lang-bash">pnpm add @copilotkit/react-core @copilotkit/react-ui
</code></pre>
<p>These two packages are all it takes to install CopilotKit in a React application. The <code>@copilotkit/react-core</code> contains core CopilotKit functionality, and the <code>@copilotkit/react-ui</code> contains some pre-built UI components for us to plug in directly.</p>
<h3 id="heading-adding-copilotkit">Adding CopilotKit</h3>
<p>There are two ways to set up CopilotKit:</p>
<ul>
<li><p><strong>Copilot Cloud</strong>: Fast, super easy to get started, and fully managed.</p>
</li>
<li><p><strong>Self-Hosting</strong>: More control but comes with extra complexity.</p>
</li>
</ul>
<p>For this tutorial, we’re going the Cloud route (because why bother with the extra complexity for now?), but feel free to self-host if you’re curious. Check out the <a target="_blank" href="https://docs.copilotkit.ai/guides/self-hosting">self-hosting guide</a> if that's your thing.</p>
<h3 id="heading-setting-up-copilot-cloud">Setting Up Copilot Cloud</h3>
<p>Here’s how you can get started with Copilot Cloud:</p>
<ul>
<li><strong>Create an account</strong></li>
</ul>
<p>Head over to the <a target="_blank" href="https://cloud.copilotkit.ai/sign-in">Copilot Cloud</a> and sign up. It's take just about a minute.</p>
<ul>
<li><strong>Get Your API Key</strong></li>
</ul>
<p>Once you're logged in, follow the steps shown on the screen to get your <strong>Copilot Cloud Public API Key</strong>. You'll also require an OpenAI API key.</p>
<p>Set your OpenAI API key, click the checkmark, and just like that, you'll get your public key.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9k3yrd5r5czgzhu6t6yq.png" alt="CopilotKit Cloud UI" /></p>
<ul>
<li><strong>Add the API Key to your</strong> <code>.env</code></li>
</ul>
<p>Update the <code>.env</code> file in the <code>ui</code> directory with your Copilot Cloud API key:</p>
<pre><code class="lang-plaintext"># 👇 ui/.env

# Rest of the env variables...
NEXT_PUBLIC_CPK_PUBLIC_API_KEY=&lt;your_copilotkit_public_key&gt;
</code></pre>
<ul>
<li><strong>Configuring the CopilotKit Provider</strong></li>
</ul>
<p>Now, to integrate <strong>CopilotKit</strong> into your application, wrap your app in the <code>CopilotKit</code> provider.</p>
<p>By wrapping our application with the provider, we ensure that the other UI components that we will be adding from the <code>@copilotkit/react-ui</code> component can interact with the CopilotKit SDK.</p>
<p>Edit the <code>ui/app/page.tsx</code> with the following lines of code:</p>
<pre><code class="lang-tsx">// 👇 ui/app/page.tsx

"use client";

// Rest of the imports...
import { CopilotKit } from "@copilotkit/react-core"; 

// Rest of the code...

export default function Home() {
  return (
    &lt;CopilotKit
      publicApiKey={process.env.NEXT_PUBLIC_CPK_PUBLIC_API_KEY}
    &gt;
      &lt;TooltipProvider&gt;
        &lt;TripsProvider&gt;
          &lt;main className="h-screen w-screen"&gt;
            &lt;MapCanvas /&gt;
          &lt;/main&gt;
        &lt;/TripsProvider&gt;
      &lt;/TooltipProvider&gt;
    &lt;/CopilotKit&gt; 
  );
}
</code></pre>
<ul>
<li><strong>CopilotKit UI Components</strong></li>
</ul>
<p>CopilotKit comes with several ready-to-use components like <code>&lt;CopilotPopup /&gt;</code>, <code>&lt;CopilotSidebar /&gt;</code>. Just place these components and they will look just wonderful.</p>
<p>If you don't want to go with the built-in components, no problem! CopilotKit also supports headless mode with <code>useCopilotChat</code>, so you can go full <strong>DIY</strong> if you're feeling creative. 😉</p>
<p>In this tutorial, we’ll use with <code>&lt;CopilotSidebar /&gt;</code> component to display the chat sidebar. However, the method stays the same with any other pre-built UI components.</p>
<p>Edit the <code>ui/app/page.tsx</code> file to include the <code>&lt;ChatSidebar /&gt;</code> component, and make sure the CSS styles are imported.</p>
<pre><code class="lang-tsx">// 👇 ui/app/page.tsx

"use client";

// Rest of the imports...

import { TasksList } from "@/components/TasksList";
import { TasksProvider } from "@/lib/hooks/use-tasks";
import { CopilotKit } from "@copilotkit/react-core";
import { CopilotSidebar } from "@copilotkit/react-ui"; 
import "@copilotkit/react-ui/styles.css"; 

// Rest of the code...

export default function Home() {
  return (
    &lt;CopilotKit
      publicApiKey={process.env.NEXT_PUBLIC_CPK_PUBLIC_API_KEY}
    &gt;
      &lt;CopilotSidebar
        defaultOpen={true}
        clickOutsideToClose={false}
        labels={{
          title: "Travel Planner",
          initial: "Hi! 👋 I'm here to plan your trips. I can help you manage your trips, add places to them, or just generally work with you to plan a new one.",
        }}
      /&gt;
      &lt;TooltipProvider&gt;
        &lt;TripsProvider&gt;
          &lt;main className="h-screen w-screen"&gt;
            &lt;MapCanvas /&gt;
          &lt;/main&gt;
        &lt;/TripsProvider&gt;
      &lt;/TooltipProvider&gt;
    &lt;/CopilotKit&gt;
  );
}
</code></pre>
<p>First, we import the required modules and custom styles to make the sidebar look great out of the box 👌. Then, drop in the <code>&lt;CopilotSidebar /&gt;</code> component.</p>
<p>In the <code>&lt;CopilotSidebar /&gt;</code> component, you can pass the labels props to change the title and the initial chat message from the AI.</p>
<p>Now, get back to your app. Look to the right, and voilà! There’s a shiny new chat sidebar all ready to be used with just a few lines of code. 😻</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r6uftdon04gnf1dmmj2r.png" alt="CopilotKit ChatSidebar" /></p>
<p>But there's something missing and that's the ability for the copilot to gain decision-making skills. We will add the feature with the help of our LangGraph in the <code>asset</code> directory.</p>
<hr />
<h2 id="heading-making-your-copilot-agentic">Making Your Copilot Agentic</h2>
<p>We’ve got a LangGraph agent running in LangGraph Studio, and a non-agentic copilot that’s functional but… not quite as smart as it could be. Let’s give the copilot some real decision-making abilities! 😎</p>
<h3 id="heading-a-quick-look-at-react-state">A Quick Look at React State</h3>
<p>Let's quickly review how the app's state works. Open up the <a target="_blank" href="https://github.com/CopilotKit/CopilotKit/blob/coagents-travel-tutorial-start/examples/coagents-travel/ui/lib/hooks/use-trips.tsx"><code>lib/hooks/use-trips.tsx</code></a> file.</p>
<p>What you’ll find here is the <code>TripsProvider</code>, which defines a lot of useful stuffs. The star of the show? The <code>state</code> object, shaped by the <code>AgentState</code> type.</p>
<p>This state is accessible throughout the app using the <code>useTrips</code> hook, which feeds components like <code>TripCard</code>, <code>TripContent</code>, and <code>TripSelect</code>.</p>
<p>If you’ve worked with React apps before, this should feel familiar — managing state through context or a library is pretty standard stuff.</p>
<h3 id="heading-merging-the-agent-with-state">Merging the Agent with State</h3>
<p>Now comes the important part: connecting our LangGraph agent to its state. To achieve this, we will set up a remote endpoint and use the <code>useCoAgent</code> hook to make the magic happen. 🌟</p>
<ul>
<li><strong>Setting Up a Tunnel</strong></li>
</ul>
<p>Remember the LangGraph Studio endpoint from earlier? You’ll need it now! If you’re using Copilot Cloud, you're all ready.</p>
<p>If you’ve gone the <strong>self-hosted</strong> route, follow the steps outlined <a target="_blank" href="https://docs.copilotkit.ai/coagents/tutorials/ai-travel-app/step-4-integrate-the-agent#integrate-the-agent">here</a></p>
<p>To connect our locally running LangGraph agent and Copilot Cloud, we'll use the CopilotKit CLI. Grab the port number of your LangGraph Studio endpoint.</p>
<blockquote>
<p>💁 <strong>LangGraph Studio Port</strong>: You'll find it at the bottom-left corner of the Studio interface.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1dpz1mznk3w2azdp2vls.png" alt="LangGraph Studio Endpoint" /></p>
<p>Now, fire up your terminal and run this command:</p>
<pre><code class="lang-bash"><span class="hljs-comment"># Replace &lt;port_number&gt; placeholder with the actual port number</span>
npx @copilotkit/cli tunnel &lt;port_number&gt;
</code></pre>
<p>Boom! You’ve created a tunnel. 🎉 Here’s what it’ll show you:</p>
<pre><code class="lang-plaintext">✔ Tunnel created successfully!  
Tunnel Information:  
Local: localhost:54209  
Public URL: https://light-pandas-argue.loca.lt  
Press Ctrl+C to stop the tunnel
</code></pre>
<p>Save that public URL. 🔖 That's going to be the gateway between our locally running LangGraph agent and CopilotKit Cloud.</p>
<blockquote>
<p>💁 Now, for the next step, we'll require <strong>LangSmith API Key</strong>. Follow this <a target="_blank" href="https://docs.smith.langchain.com/administration/how_to_guides/organization_management/create_account_api_key#api-keys">guide</a> to get one.</p>
</blockquote>
<ul>
<li><strong>Connecting the Tunnel to Copilot Cloud</strong></li>
</ul>
<p>Head over to <a target="_blank" href="https://cloud.copilotkit.ai">Copilot Cloud</a>, scroll to the <strong>Remote Endpoints</strong> section, and hit the <code>+ Add New</code> button.</p>
<ul>
<li><p>Select the <code>LangGraph</code> platform.</p>
</li>
<li><p>Add the public URL. (the one generated from the CopilotKit CLI) and add your LangSmith API Key.</p>
</li>
<li><p>Click <strong>Create.</strong></p>
</li>
</ul>
<p>🎉 Done! Your agent’s endpoint is now listed, and CopilotKit knows exactly where to send requests when the agent is called.</p>
<ul>
<li><strong>Locking the agent</strong></li>
</ul>
<p>Since, we only have one single agent here, let's make sure to lock the <code>&lt;CopilotKit /&gt;</code> provider to lock all the requests to this specific agent. To add the agent, simply modify the props to include the name of the <code>agent</code>.</p>
<blockquote>
<p>💁 Curious about handling multiple agents? Check out out the <a target="_blank" href="https://docs.copilotkit.ai/coagents/concepts/multi-agent-flows">multi-agent concept guide</a>.</p>
</blockquote>
<pre><code class="lang-tsx">// 👇 ui/app/page.tsx

// Rest of the code...
&lt;CopilotKit
  // Rest of the code...
  agent="travel"
&gt;
    {/* Rest of the code... */}
&lt;/CopilotKit&gt;
</code></pre>
<p>We provide the name of the agent as <code>travel</code> because we have already defined it in the <a target="_blank" href="https://github.com/CopilotKit/CopilotKit/blob/main/examples/coagents-travel/agent/langgraph.json">agents/langgraph.json</a> file.</p>
<p>And just like this, the copilot is now truly agentic. It can not only hold conversations but also make decisions. Pretty cool, right? 🤯</p>
<h3 id="heading-hooking-up-the-agent-and-state">Hooking Up the Agent and State</h3>
<p>At this point, we want to connect the LangGraph agent's state with our app's state. This will allow us to add real-time dynamic interactions!</p>
<p>LangGraph agents keep track of their own state, as you've already seen it in the LangGraph studio's bottom left corner.</p>
<blockquote>
<p>🤔 <strong>Now what's the idea?</strong></p>
</blockquote>
<p>We want to achieve a two-way connection between these states. For this, we have our friendly hook <code>useCoAgent</code> from CopilotKit that helps us achieve just that.</p>
<p>Edit the <code>ui/lib/hooks/use-trips.tsx</code> file with the following lines of code to add the <code>useCoAgent</code> hook.</p>
<pre><code class="lang-tsx">// 👇 ui/lib/hooks/use-trips.tsx

// Rest of the imports...
import { AgentState, defaultTrips} from "@/lib/trips"; 
import { useCoAgent } from "@copilotkit/react-core"; 

export const TripsProvider = ({ children }: { children: ReactNode }) =&gt; {
  const { state, setState } = useCoAgent&lt;AgentState&gt;({
    name: "travel",
    initialState: {
      trips: defaultTrips,
      selected_trip_id: defaultTrips[0].id,
    },
  });

  // Rest of the code...
</code></pre>
<p>Yeah, you saw it right. That's all it takes to sync the two-way states. 😯</p>
<p>Now, let's break down the code line by line:</p>
<blockquote>
<p>💡 The <code>useCoAgent</code> hook is generic, which means you can specify a type that mirrors your LangGraph agent’s state. In this example, we use <code>AgentState</code> to keep things a bit consistent. You can get away with typecasting it as <code>any</code>, but that's not a good practice in general. So avoid doing that most of the time.</p>
</blockquote>
<p>The <code>name</code> parameter ties everything back to your graph’s name from <code>agent/langgraph.json</code>. Make sure to name it correctly, as it ensures the agent and our app are always on the same page.</p>
<p>For the <code>initialState</code>, we use <code>defaultTrips</code> (though it is not required) from <code>@/lib/types.ts</code>.</p>
<p>We add some initial trips, just so we can test the working of it right away.</p>
<p>This is how the initial state, i.e., <code>defaultTrips</code>, is arranged:</p>
<pre><code class="lang-typescript"><span class="hljs-comment">// 👇 ui/lib/types.ts</span>

<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> defaultTrips: Trip[] = [
  {
    id: <span class="hljs-string">"1"</span>,
    name: <span class="hljs-string">"Business Trip to NYC"</span>,
    center_latitude: <span class="hljs-number">40.7484</span>,
    center_longitude: <span class="hljs-number">-73.9857</span>,
    places: [
      {
        id: <span class="hljs-string">"1"</span>,
        name: <span class="hljs-string">"Central Park"</span>,
        address: <span class="hljs-string">"New York, NY 10024"</span>,
        description: <span class="hljs-string">"A famous park in New York City"</span>,
        latitude: <span class="hljs-number">40.785091</span>,
        longitude: <span class="hljs-number">-73.968285</span>,
        rating: <span class="hljs-number">4.7</span>,
      },
      {
        id: <span class="hljs-string">"3"</span>,
        name: <span class="hljs-string">"Times Square"</span>,
        address: <span class="hljs-string">"Times Square, New York, NY 10036"</span>,
        description: <span class="hljs-string">"A famous square in New York City"</span>,
        latitude: <span class="hljs-number">40.755499</span>,
        longitude: <span class="hljs-number">-73.985701</span>,
        rating: <span class="hljs-number">4.6</span>,
      },
    ],
    zoom_level: <span class="hljs-number">14</span>,
  },
  <span class="hljs-comment">// Rest of the trips...</span>
];
</code></pre>
<h3 id="heading-time-to-test-it">Time to Test it!</h3>
<p>Fire up your app and ask the Copilot something about your trips.</p>
<pre><code class="lang-plaintext">How many trips do I have?
</code></pre>
<p>See how the agent pulls the data from the app's state? Magic, isn't it? 😻</p>
<p>The state is shared between the application and the agent, so try deleting/editing a trip manually, ask the question again, and it should reply accordingly:</p>
<pre><code class="lang-plaintext">What trips do I have now?
</code></pre>
<p>The agent knows what’s up. Better yet, you can give the agent tasks directly:</p>
<pre><code class="lang-plaintext">Add some hotels to my Paris trip
</code></pre>
<p>And voilà! The state updates, and your UI reflects the changes.</p>
<p>The core functionality of the app is done for now. We just need to improve the user experience by adding streaming text and other features to provide real-time responses to the user.</p>
<hr />
<h2 id="heading-stream-the-response-in-the-ui">Stream the Response in the UI</h2>
<p>Now that we can interact with our agent, retrieve and update data, why not level up the user experience by adding a text stream feature?</p>
<p>It's like watching real-time progress while the agent works, similar to how you see many other popular AIs like <code>ChatGPT</code> respond.</p>
<p>In this step, we’ll be implementing the <code>copilotkit_emit_state</code> SDK function within our LangGraph agent, so the agent will emit progress as it’s working. 🔥</p>
<h3 id="heading-installing-the-copilotkit-sdk">Installing the CopilotKit SDK</h3>
<p>First things first, let’s get the <strong>CopilotKit SDK</strong> installed. Since we’re working with a Python-based agent here (and managing it with <strong>poetry</strong>), we’ll install the Python SDK.</p>
<blockquote>
<p>💁 Not sure how to install poetry? You can find the installation guide <a target="_blank" href="https://python-poetry.org/docs/#installation">here</a>.</p>
</blockquote>
<pre><code class="lang-bash">poetry add copilotkit==0.1.31a4
</code></pre>
<p>Since, we'll be editing the <code>search_node</code>, we'll jump right into the <code>search.py</code> file.</p>
<h3 id="heading-manually-emitting-the-agents-state">Manually emitting the agent’s state</h3>
<p>With CoAgents, the agent’s state is emitted when a node changes (i.e., when an edge is traversed). But what if we want to display progress in the middle of an action? Great news! We can manually emit the state using <code>copilotkit_emit_state</code>.</p>
<p>Let’s add a custom configuration to the <code>search_node</code> so that we can emit the intermediate state.</p>
<p>Open up <code>agent/travel/search.py</code> and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 agent/travel/search.py</span>

<span class="hljs-comment"># Rest of the imports...</span>
<span class="hljs-keyword">from</span> copilotkit.langchain <span class="hljs-keyword">import</span> copilotkit_emit_state, copilotkit_customize_config 

<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">search_node</span>(<span class="hljs-params">state: AgentState, config: RunnableConfig</span>):</span>
    <span class="hljs-string">"""
    The search node is responsible for searching the for places.
    """</span>
    ai_message = cast(AIMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-1</span>])


    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[{
            <span class="hljs-string">"state_key"</span>: <span class="hljs-string">"search_progress"</span>,
            <span class="hljs-string">"tool"</span>: <span class="hljs-string">"search_for_places"</span>,
            <span class="hljs-string">"tool_argument"</span>: <span class="hljs-string">"search_progress"</span>,
        }],
    )

    <span class="hljs-comment"># Rest of the code...</span>
</code></pre>
<h3 id="heading-emitting-the-intermediate-state">Emitting the intermediate state</h3>
<p>Now, let’s use <code>copilotkit_emit_state</code> to emit the state manually as the search progresses. You’ll see updates for every query we send.</p>
<p>Let’s edit <code>agent/travel/search.py</code> again to emit state at the beginning of our search and as results come in.</p>
<p>Edit the <code>agent/travel/search.py</code> with the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 agent/travel/search.py</span>

<span class="hljs-comment"># Rest of the code...</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">search_node</span>(<span class="hljs-params">state: AgentState, config: RunnableConfig</span>):</span>
    <span class="hljs-string">"""
    The search node is responsible for searching the for places.
    """</span>
    ai_message = cast(AIMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-1</span>])

    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[{
            <span class="hljs-string">"state_key"</span>: <span class="hljs-string">"search_progress"</span>,
            <span class="hljs-string">"tool"</span>: <span class="hljs-string">"search_for_places"</span>,
            <span class="hljs-string">"tool_argument"</span>: <span class="hljs-string">"search_progress"</span>,
        }],
    )

    <span class="hljs-comment"># ^ Previous code</span>

    state[<span class="hljs-string">"search_progress"</span>] = state.get(<span class="hljs-string">"search_progress"</span>, [])
    queries = ai_message.tool_calls[<span class="hljs-number">0</span>][<span class="hljs-string">"args"</span>][<span class="hljs-string">"queries"</span>]

    <span class="hljs-keyword">for</span> query <span class="hljs-keyword">in</span> queries:
        state[<span class="hljs-string">"search_progress"</span>].append({
            <span class="hljs-string">"query"</span>: query,
            <span class="hljs-string">"results"</span>: [],
            <span class="hljs-string">"done"</span>: <span class="hljs-literal">False</span>
        })

    <span class="hljs-keyword">await</span> copilotkit_emit_state(config, state) 

    <span class="hljs-comment"># Rest of the code...</span>
</code></pre>
<h3 id="heading-updating-and-emitting-progress">Updating and Emitting Progress</h3>
<p>Now it’s time to show the results in real-time. As the search completes, we’ll update the progress.</p>
<p>Finally, update the <code>agent/travel/search.py</code> with the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 agent/travel/search.py</span>

<span class="hljs-comment"># Rest of the code...</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">search_node</span>(<span class="hljs-params">state: AgentState, config: RunnableConfig</span>):</span>
    <span class="hljs-string">"""
    The search node is responsible for searching the for places.
    """</span>
    ai_message = cast(AIMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-1</span>])

    config = copilotkit_customize_config(
        config,
        emit_intermediate_state=[{
            <span class="hljs-string">"state_key"</span>: <span class="hljs-string">"search_progress"</span>,
            <span class="hljs-string">"tool"</span>: <span class="hljs-string">"search_for_places"</span>,
            <span class="hljs-string">"tool_argument"</span>: <span class="hljs-string">"search_progress"</span>,
        }],
    )

    state[<span class="hljs-string">"search_progress"</span>] = state.get(<span class="hljs-string">"search_progress"</span>, [])
    queries = ai_message.tool_calls[<span class="hljs-number">0</span>][<span class="hljs-string">"args"</span>][<span class="hljs-string">"queries"</span>]

    <span class="hljs-keyword">for</span> query <span class="hljs-keyword">in</span> queries:
        state[<span class="hljs-string">"search_progress"</span>].append({
            <span class="hljs-string">"query"</span>: query,
            <span class="hljs-string">"results"</span>: [],
            <span class="hljs-string">"done"</span>: <span class="hljs-literal">False</span>
        })

    <span class="hljs-keyword">await</span> copilotkit_emit_state(config, state) 

    <span class="hljs-comment"># ^ Previous code</span>

    places = []
    <span class="hljs-keyword">for</span> i, query <span class="hljs-keyword">in</span> enumerate(queries):
        response = gmaps.places(query)
        <span class="hljs-keyword">for</span> result <span class="hljs-keyword">in</span> response.get(<span class="hljs-string">"results"</span>, []):
            place = {
                <span class="hljs-string">"id"</span>: result.get(<span class="hljs-string">"place_id"</span>, <span class="hljs-string">f"<span class="hljs-subst">{result.get(<span class="hljs-string">'name'</span>, <span class="hljs-string">''</span>)}</span>-<span class="hljs-subst">{i}</span>"</span>),
                <span class="hljs-string">"name"</span>: result.get(<span class="hljs-string">"name"</span>, <span class="hljs-string">""</span>),
                <span class="hljs-string">"address"</span>: result.get(<span class="hljs-string">"formatted_address"</span>, <span class="hljs-string">""</span>),
                <span class="hljs-string">"latitude"</span>: result.get(<span class="hljs-string">"geometry"</span>, {}).get(<span class="hljs-string">"location"</span>, {}).get(<span class="hljs-string">"lat"</span>, <span class="hljs-number">0</span>),
                <span class="hljs-string">"longitude"</span>: result.get(<span class="hljs-string">"geometry"</span>, {}).get(<span class="hljs-string">"location"</span>, {}).get(<span class="hljs-string">"lng"</span>, <span class="hljs-number">0</span>),
                <span class="hljs-string">"rating"</span>: result.get(<span class="hljs-string">"rating"</span>, <span class="hljs-number">0</span>),
            }
            places.append(place)
        state[<span class="hljs-string">"search_progress"</span>][i][<span class="hljs-string">"done"</span>] = <span class="hljs-literal">True</span>
        <span class="hljs-keyword">await</span> copilotkit_emit_state(config, state) 

    state[<span class="hljs-string">"search_progress"</span>] = []
    <span class="hljs-keyword">await</span> copilotkit_emit_state(config, state) 

    <span class="hljs-comment"># Rest of the code...</span>
</code></pre>
<h3 id="heading-rendering-the-progress-in-the-ui">Rendering the Progress in the UI</h3>
<p>To show the progress in the UI, we’ll use the <a target="_blank" href="https://docs.copilotkit.ai/reference/hooks/useCoAgentStateRender">useCoAgentStateRender</a> hook. This hook will conditionally render the <code>search_progress</code> state.</p>
<p>All we need to do is tell CopilotKit to conditionally render the <code>search_progress</code> state key through the <code>useCoAgentStateRender</code> hook.</p>
<p>Now let’s modify <code>ui/lib/hooks/use-trips.tsx</code> to display the search progress:</p>
<pre><code class="lang-tsx">// 👇 ui/lib/hooks/use-trips.tsx

// Rest of the imports...
import { useCoAgent, useCoAgentStateRender } from "@copilotkit/react-core"; 
import { SearchProgress } from "@/components/SearchProgress"; 

export const TripsProvider = ({ children }: { children: ReactNode }) =&gt; {
  // Rest of the code...

  useCoAgentStateRender&lt;AgentState&gt;({
    name: "travel",
    render: ({ state }) =&gt; {
      if (state.search_progress) {
        return &lt;SearchProgress progress={state.search_progress} /&gt;
      }
      return null;
    },
  });

  // Rest of the code...
}
</code></pre>
<p>The <code>&lt;SearchProgress /&gt;</code> component is already set up for you. If you're curious, feel free to check out the implementation in <code>ui/components/SearchProgress.tsx</code>. 🙌</p>
<blockquote>
<p>💁 <strong>Bonus</strong>: The <code>search_progress</code> state key is pre-defined in the <code>AgentState</code> type in <code>ui/lib/types.ts</code>, so no need to worry about creating it from scratch!</p>
</blockquote>
<p>Now, give it a spin! Ask the agent questions and you’ll see the progress update in real time. 🤩</p>
<hr />
<h2 id="heading-add-control-by-adding-human-in-the-loop">Add control by adding Human in the Loop</h2>
<p>Alright, time for some real-world magic. What if the agent is about to make a decision you don’t agree with?</p>
<p><a target="_blank" href="https://docs.copilotkit.ai/coagents/concepts/human-in-the-loop">Human in the Loop (HITL)</a> lets you approve, reject, or modify the actions the agent wants to perform.</p>
<p>In this step, we’ll set up a "breakpoint" in the agent's flow, forcing it to pause and wait for your approval before continuing.</p>
<blockquote>
<p>💁 Curious about breakpoints? You can learn more <a target="_blank" href="https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/breakpoints/?h=breakp">here</a>.</p>
</blockquote>
<p>Once the breakpoint is hit, we’ll send it to the front end, where the user will approve or reject the action. The agent will then continue based on the user’s decision. All together, this process will look like this:</p>
<blockquote>
<p>Check out the infographic below to see how the whole flow works: 👇</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8lnzbbkezpd05rf1pki2.png" alt="Coagents HITL Infographic" /></p>
<ul>
<li><strong>Adding a Breakpoint for Human-in-the-Loop</strong></li>
</ul>
<p>With our LangGraph, adding human-in-the-loop functionality is straightforward. The <code>trips_node</code> functions as an intermediary to the <code>perform_trips_node</code>, which lets us pause execution at the <code>trips_node</code> by setting a breakpoint.</p>
<p>In the <code>agent/travel/agent.py</code> file, we just need to tell the agent where to hit that pause. We’re doing this in the <code>compile</code> function:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 agent/travel/agent.py</span>

<span class="hljs-comment"># Rest of the code...</span>

graph = graph_builder.compile(
    checkpointer=MemorySaver(),
    <span class="hljs-comment"># Pause right here and wait for the user!</span>
    interrupt_after=[<span class="hljs-string">"trips_node"</span>], 
)
</code></pre>
<p>Now, just by doing this, the agent will ask, "Should I proceed?" instead of blindly continuing with the work.</p>
<ul>
<li><strong>Handling the User's Decision</strong></li>
</ul>
<p>When the user hits that pause button, we need to check what they want to do. Did they approve the action? Or did they hit "Cancel" and start over?</p>
<p>In <code>perform_trips_node</code>, we’ll grab the tool message and check what the user decided:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 agent/travel/trips.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">perform_trips_node</span>(<span class="hljs-params">state: AgentState, config: RunnableConfig</span>):</span>
    <span class="hljs-string">"""Execute trip operations"""</span>
    ai_message = cast(AIMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-2</span>]) 
    tool_message = cast(ToolMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-1</span>]) 

    <span class="hljs-comment"># Rest of the code...</span>
</code></pre>
<p>Now, here, the conditional will check the user's decision and act accordingly.</p>
<p>If the user says "Cancel," we stop everything and simply return with the custom message. Otherwise, for any other response, it will continue with the work.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 agent/travel/trips.py</span>

<span class="hljs-comment"># Rest of the code...</span>
<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">perform_trips_node</span>(<span class="hljs-params">state: AgentState, config: RunnableConfig</span>):</span>
    <span class="hljs-string">"""Execute trip operations"""</span>
    ai_message = cast(AIMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-2</span>])
    tool_message = cast(ToolMessage, state[<span class="hljs-string">"messages"</span>][<span class="hljs-number">-1</span>])


    <span class="hljs-keyword">if</span> tool_message.content == <span class="hljs-string">"CANCEL"</span>:
      <span class="hljs-keyword">return</span> {
        <span class="hljs-string">"messages"</span>: AIMessage(content=<span class="hljs-string">"Cancelled operation of trip."</span>),
      }

    <span class="hljs-comment"># handle the edge case where the AI message is not an AIMessage or does not have tool calls, should never happen.</span>
    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> isinstance(ai_message, AIMessage) <span class="hljs-keyword">or</span> <span class="hljs-keyword">not</span> ai_message.tool_calls:
        <span class="hljs-keyword">return</span> state

    <span class="hljs-comment"># Rest of the code...</span>
</code></pre>
<ul>
<li><strong>Rendering the Decision UI</strong></li>
</ul>
<p>Now, it's time to update the front-end to render the tool calls and capture the user's decision, passing it back to the agent. For this, we'll use <code>useCopilotAction</code> hooks for each tool call with the <code>renderAndWait</code> option.</p>
<p>Edit the <code>ui/lib/hooks/use-trips.tsx</code> with the following lines of code:</p>
<pre><code class="lang-tsx">// 👇 ui/lib/hooks/use-trips.tsx

// Rest of the imports...
import { AddTrips, EditTrips, DeleteTrips } from "@/components/humanInTheLoop"; 
import { useCoAgent, useCoAgentStateRender, useCopilotAction } from "@copilotkit/react-core"; 

// Rest of the code...

export const TripsProvider = ({ children }: { children: ReactNode }) =&gt; {
  // Rest of the code...

  useCoAgentStateRender&lt;AgentState&gt;({
    name: "travel",
    render: ({ state }) =&gt; {
      return &lt;SearchProgress progress={state.search_progress} /&gt;
    },
  });


  useCopilotAction({ 
    name: "add_trips",
    description: "Add some trips",
    parameters: [
      {
        name: "trips",
        type: "object[]",
        description: "The trips to add",
        required: true,
      },
    ],
    renderAndWait: AddTrips,
  });

  useCopilotAction({
    name: "update_trips",
    description: "Update some trips",
    parameters: [
      {
        name: "trips",
        type: "object[]",
        description: "The trips to update",
        required: true,
      },
    ],
    renderAndWait: EditTrips,
  });

  useCopilotAction({
    name: "delete_trips",
    description: "Delete some trips",
    parameters: [
      {
        name: "trip_ids",
        type: "string[]",
        description: "The ids of the trips to delete",
        required: true,
      },
    ],
    renderAndWait: (props) =&gt; DeleteTrips({ ...props, trips: state.trips }),
  });

  // Rest of the code...
</code></pre>
<p>With this setup, the front-end is ready to render the tool calls and capture the user's decision. However, there’s one important thing we haven’t yet covered: how we handle the user’s input and send it back to the agent?</p>
<ul>
<li><strong>Optional: Understanding the</strong> <code>humanInTheLoop</code> components</li>
</ul>
<p>Let’s take a quick dive into how we handle this in the front-end. We'll use the <code>DeleteTrips</code> component as an example, but the same logic applies to <code>AddTrips</code> and <code>EditTrips</code>.</p>
<pre><code class="lang-tsx">// 👇 ui/lib/components/humanInTheLoop/DeleteTrips.tsx

import { Trip } from "@/lib/types";
import { PlaceCard } from "@/components/PlaceCard";
import { X, Trash } from "lucide-react";
import { ActionButtons } from "./ActionButtons"; 
import { RenderFunctionStatus } from "@copilotkit/react-core";

export type DeleteTripsProps = {
  args: any;
  status: RenderFunctionStatus;
  handler: any;
  trips: Trip[];
};

export const DeleteTrips = ({ args, status, handler, trips }: DeleteTripsProps) =&gt; {
  const tripsToDelete = trips.filter((trip: Trip) =&gt; args?.trip_ids?.includes(trip.id));

  return (
    &lt;div className="space-y-4 w-full bg-secondary p-6 rounded-lg"&gt;
    &lt;h1 className="text-sm"&gt;The following trips will be deleted:&lt;/h1&gt;
      {status !== "complete" &amp;&amp; tripsToDelete?.map((trip: Trip) =&gt; (
        &lt;div key={trip.id} className="flex flex-col gap-4"&gt;
          &lt;&gt;
            &lt;hr className="my-2" /&gt;
            &lt;div className="flex flex-col gap-4"&gt;
            &lt;h2 className="text-lg font-bold"&gt;{trip.name}&lt;/h2&gt;
            {trip.places?.map((place) =&gt; (
              &lt;PlaceCard key={place.id} place={place} /&gt;
            ))}
            &lt;/div&gt;
          &lt;/&gt;
        &lt;/div&gt;
      ))}
      { status !== "complete" &amp;&amp; (

        &lt;ActionButtons
          status={status} 
          handler={handler} 
          approve={&lt;&gt;&lt;Trash className="w-4 h-4 mr-2" /&gt; Delete&lt;/&gt;} 
          reject={&lt;&gt;&lt;X className="w-4 h-4 mr-2" /&gt; Cancel&lt;/&gt;} 
        /&gt;
      )}
    &lt;/div&gt;
  );
};
</code></pre>
<p>The crucial part here is the <code>ActionButtons</code> component, which allows the user to approve or reject the action. This is how we capture the user's decision:</p>
<pre><code class="lang-tsx">// 👇 ui/lib/components/humanInTheLoop/ActionButtons.tsx

import { RenderFunctionStatus } from "@copilotkit/react-core";
import { Button } from "../ui/button";

export type ActionButtonsProps = {
    status: RenderFunctionStatus;
    handler: any;
    approve: React.ReactNode;
    reject: React.ReactNode;
}

export const ActionButtons = ({ status, handler, approve, reject }: ActionButtonsProps) =&gt; (
  &lt;div className="flex gap-4 justify-between"&gt;
    &lt;Button 
      className="w-full"
      variant="outline"
      disabled={status === "complete" || status === "inProgress"} 
      onClick={() =&gt; handler?.("CANCEL")} 
    &gt;
      {reject}
    &lt;/Button&gt;
    &lt;Button 
      className="w-full"
      disabled={status === "complete" || status === "inProgress"} 
      onClick={() =&gt; handler?.("SEND")} 
    &gt;
      {approve}
    &lt;/Button&gt;
  &lt;/div&gt;
);
</code></pre>
<p>The buttons here call the <code>handler?.("CANCEL")</code> when the user clicks "Cancel" and <code>handler?.("SEND")</code> when they click "Delete." This sends the user's decision (either "CANCEL" or "SEND") back to the agent.</p>
<p>The important piece here is that the <code>onClick</code> handlers emit the user's decision back to the agent.</p>
<blockquote>
<p>💡 If you want to allow the user to edit the tool call arguments before sending them back to the agent, you can achieve this by modifying the <code>onClick</code> handlers and adjusting the way the agent processes the tool calls.</p>
</blockquote>
<p>That's all. 😮‍💨 We've successfully added human-in-the-loop functionality. Now it can prompt the user to approve or reject actions like adding, editing, or deleting trips, and have those decisions sent back to the agent.</p>
<hr />
<h2 id="heading-conclusion">Conclusion ⚡</h2>
<p>We covered a lot in this tutorial. 😴 Hopefully, you got to know how to add an agentic copilot to your application with CopilotKit and also learned how to perform state changes in real-time and implement the human-in-the-loop concept.</p>
<p>Complete Source Code: <a target="_blank" href="https://github.com/CopilotKit/CopilotKit/tree/main/examples/coagents-travel">Source Code</a></p>
<blockquote>
<p>💁 You can also refer to the <a target="_blank" href="https://docs.copilotkit.ai/coagents/tutorials/ai-travel-app">original documentation</a> of this project.</p>
</blockquote>
<p>Thank you so much for reading! 🎉 🫡</p>
<blockquote>
<p>Share your thoughts in the comment section below! 👇</p>
</blockquote>
<p><img src="https://i.giphy.com/media/v1.Y2lkPTc5MGI3NjExYm9lNGducHVjM3lqZzI4Y3VmdWxkZzlldGQ1bWdpOXNkaHlkYzc1ZCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/aNqEFrYVnsS52/giphy.gif" alt="Cat Typing" /></p>
]]></content:encoded></item><item><title><![CDATA[How to Sync Your Portfolio with GitHub Repositories 😻👌]]></title><description><![CDATA[Learn to sync your portfolio site with GitHub using GitHub Actions for automated project showcasing]]></description><link>https://shricodev.hashnode.dev/how-to-sync-your-portfolio-with-github-repositories</link><guid isPermaLink="true">https://shricodev.hashnode.dev/how-to-sync-your-portfolio-with-github-repositories</guid><category><![CDATA[software development]]></category><category><![CDATA[GitHub]]></category><category><![CDATA[Programming Blogs]]></category><category><![CDATA[github-actions]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Wed, 22 Jan 2025 02:46:40 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1737513783771/610e65ae-9aa9-486d-af85-3385f58dfa9b.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-the-issue">The Issue 🫤</h2>
<p>We all (at least most of us) have a portfolio site. As developers, we want to showcase the projects we’ve worked on, which we mostly store in our GitHub profiles.</p>
<blockquote>
<p><strong>🤔 But how do we go about showing them on our portfolio site?</strong></p>
</blockquote>
<p>Previously, I used to have a <code>ts</code> file to store the links to GitHub repositories and card information, like this:</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> data = [
  {
    id: <span class="hljs-built_in">Math</span>.random(),
    subtitle:
      <span class="hljs-string">"A Resume Builder with OpenAI that can generate a resume from given data. Supports file upload with Supabase."</span>,
    title: <span class="hljs-string">"Resume Builder with OpenAI"</span>,
    github: <span class="hljs-string">"https://github.com/shricodev/resume-builder-openai"</span>,
    noLive: <span class="hljs-literal">true</span>,
  },
  {
    id: <span class="hljs-built_in">Math</span>.random(),
    title: <span class="hljs-string">"PDFWhisper - OpenAI"</span>,
    subtitle: <span class="hljs-string">"A PDF Chatbot with OpenAI. It can answer questions from a PDF."</span>,
    github: <span class="hljs-string">"https://github.com/shricodev/pdfwhisper-openai"</span>,
    liveUrl: <span class="hljs-string">"https://pdfwhisper-openai.vercel.app/"</span>,
  },
  <span class="hljs-comment">// ...Rest of the project info</span>
];
</code></pre>
<p>Alternatively, you might use a <code>json</code> file to store similar info.</p>
<h3 id="heading-the-problem-with-this-approach">The Problem with This Approach 🤨</h3>
<p>The real problem is that this process is entirely manual. You also can’t display project content stored in your README files. If you build a new project, you’d need to manually edit the file, commit the changes, and push them to your repository. Ugh, such a work 😮‍💨.</p>
<p>Why not automate this and, optionally, show <strong>README</strong> content in markdown as well?</p>
<hr />
<h2 id="heading-the-fix">The Fix 🛠️</h2>
<p>I was frustrated with this process and always felt my portfolio site was outdated because of it. The first thing that crossed my mind was: <strong>Can we use GitHub Actions to automate it?</strong></p>
<p>And the answer was <strong>yes</strong>... But how?</p>
<p>There are a few ways to solve this problem. You could use JavaScript libraries built to interact with the GitHub API (GitHub recommends <a target="_blank" href="https://docs.github.com/en/rest/guides/scripting-with-the-rest-api-and-javascript?apiVersion=2022-11-28">Octokit.js</a>) or find other alternatives.</p>
<p>When I read the docs, it mentioned generating a personal access token, which felt unnecessary since I only needed data from public repositories. After all, GitHub has a public API (<a target="_blank" href="https://api.github.com">https://api.github.com</a>) with generous rate limits, perfect for fetching repository info.</p>
<p>So, I decided to ditch libraries and use some <strong>Bash</strong> magic instead. 🪄🧙</p>
<h3 id="heading-fetching-data-directly-with-curl">Fetching Data Directly with <code>curl</code></h3>
<p>Using a simple <code>curl</code> command, I fetched all the names of the repository:</p>
<pre><code class="lang-bash">curl -s <span class="hljs-string">"https://api.github.com/users/shricodev/repos?per_page=300"</span> | jq -r <span class="hljs-string">'.[] | select(.private == false and .fork == false) | .name'</span> | uniq
</code></pre>
<p>This command fetches all repositories names for a given user. It filters out forked repositories (In most cases, definitely don’t want to show any forked repositories in your portfolio) using <code>jq</code>, and then filters the project based on uniqueness (just to be on safer side).</p>
<blockquote>
<p>🤔 <strong>But how do we filter specific repositories to show in our site?</strong></p>
</blockquote>
<p>For this, what we will do is add a unique topic like <code>showcase</code> to the repositories you want to display (you can name it as you like but add this unique topic to all the repositories you wish to show in your portfolio).</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1737513226361/156bc324-2d4f-4b34-bf56-144320163947.png" alt class="image--center mx-auto" /></p>
<p>Now, all we need to do is modify that above command <code>jq</code> filtering to check if the topic <code>showcase</code> is in the repository, if it is then only fetch all the metadata and stuffs for that repository</p>
<pre><code class="lang-bash">curl -s <span class="hljs-string">"https://api.github.com/users/shricodev/repos?per_page=300"</span> | jq -r <span class="hljs-string">'.[] | select(.private == false and .fork == false and (.topics | index("showcase") != null)) | .name'</span> | uniq
</code></pre>
<p>Next, I generated a personal access token (PAT) from GitHub’s Developer Settings page and added it to the repository’s Action secrets (Secrets and variables -&gt; Actions).</p>
<p>Here’s how that looks:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1737513328921/e26166ce-41ec-479f-a278-a8a036ed7280.png" alt class="image--center mx-auto" /></p>
<h3 id="heading-writing-the-github-action">Writing the GitHub Action ✍️</h3>
<p>Here’s the complete GitHub Action workflow that uses the above <code>curl</code> command:</p>
<pre><code class="lang-yaml"><span class="hljs-attr">name:</span> <span class="hljs-string">Fetch</span> <span class="hljs-string">Public</span> <span class="hljs-string">Repos</span> <span class="hljs-string">README's</span>

<span class="hljs-attr">on:</span>
  <span class="hljs-attr">schedule:</span>
    <span class="hljs-comment"># Run once every single day.</span>
    <span class="hljs-bullet">-</span> <span class="hljs-attr">cron:</span> <span class="hljs-string">'0 0 * * *'</span>
  <span class="hljs-attr">workflow_dispatch:</span>

<span class="hljs-attr">permissions:</span>
  <span class="hljs-attr">contents:</span> <span class="hljs-string">write</span>

<span class="hljs-attr">jobs:</span>
  <span class="hljs-attr">fetch-readmes:</span>
    <span class="hljs-attr">runs-on:</span> <span class="hljs-string">ubuntu-latest</span>

    <span class="hljs-attr">steps:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Checkout</span> <span class="hljs-string">the</span> <span class="hljs-string">repository</span>
        <span class="hljs-attr">uses:</span> <span class="hljs-string">actions/checkout@v4</span>

      <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Fetch</span> <span class="hljs-string">repository</span> <span class="hljs-string">list</span> <span class="hljs-string">and</span> <span class="hljs-string">README's</span> <span class="hljs-string">with</span> <span class="hljs-string">metadata</span>
        <span class="hljs-attr">env:</span>
          <span class="hljs-attr">TARGET_USER:</span> <span class="hljs-string">&lt;username&gt;</span>
          <span class="hljs-attr">TARGET_FOLDER:</span> <span class="hljs-string">'&lt;path_to_the_folder_to_store_readme(s)&gt;'</span>
        <span class="hljs-attr">run:</span> <span class="hljs-string">|
          mkdir -p "$TARGET_FOLDER"
</span>
          <span class="hljs-comment"># Fetch list of public repos</span>
          <span class="hljs-string">repos=$(curl</span> <span class="hljs-string">-s</span> <span class="hljs-string">"https://api.github.com/users/$TARGET_USER/repos?per_page=300"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.[] | select(.private == false and .fork == false and (.topics | index("showcase") != null)) | .name'</span> <span class="hljs-string">|</span> <span class="hljs-string">uniq)</span>

          <span class="hljs-string">for</span> <span class="hljs-string">repo</span> <span class="hljs-string">in</span> <span class="hljs-string">$repos;</span> <span class="hljs-string">do</span>
            <span class="hljs-string">readme_file="$TARGET_FOLDER/${repo}_README.mdx"</span>

            <span class="hljs-string">if</span> [[ <span class="hljs-string">-f</span> <span class="hljs-string">"$readme_file"</span> ]]<span class="hljs-string">;</span> <span class="hljs-string">then</span>
              <span class="hljs-string">echo</span> <span class="hljs-string">"README for $repo already exists, skipping..."</span>
              <span class="hljs-string">continue</span>
            <span class="hljs-string">fi</span>

            <span class="hljs-string">repo_metadata=$(curl</span> <span class="hljs-string">-s</span> <span class="hljs-string">"https://api.github.com/repos/$TARGET_USER/$repo"</span><span class="hljs-string">)</span>
            <span class="hljs-string">title=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.name'</span><span class="hljs-string">)</span>
            <span class="hljs-string">description=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.description // ""'</span><span class="hljs-string">)</span>
            <span class="hljs-string">clone_url=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.clone_url'</span><span class="hljs-string">)</span>
            <span class="hljs-string">language=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.language // ""'</span><span class="hljs-string">)</span>
            <span class="hljs-string">homepage=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.homepage // ""'</span><span class="hljs-string">)</span>
            <span class="hljs-string">topics=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.topics | join(", ")'</span><span class="hljs-string">)</span>
            <span class="hljs-string">created_at=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.created_at'</span><span class="hljs-string">)</span>
            <span class="hljs-string">updated_at=$(echo</span> <span class="hljs-string">"$repo_metadata"</span> <span class="hljs-string">|</span> <span class="hljs-string">jq</span> <span class="hljs-string">-r</span> <span class="hljs-string">'.updated_at'</span><span class="hljs-string">)</span>

            <span class="hljs-string">status_code=$(curl</span> <span class="hljs-string">-o</span> <span class="hljs-string">/dev/null</span> <span class="hljs-string">-s</span> <span class="hljs-string">-w</span> <span class="hljs-string">"<span class="hljs-template-variable">%{http_code}</span>"</span> <span class="hljs-string">https://raw.githubusercontent.com/$TARGET_USER/$repo/main/README.md)</span>

            <span class="hljs-string">if</span> [ <span class="hljs-string">"$status_code"</span> <span class="hljs-string">-eq</span> <span class="hljs-number">200</span> ]<span class="hljs-string">;</span> <span class="hljs-string">then</span>
              <span class="hljs-string">readme_content=$(curl</span> <span class="hljs-string">-s</span> <span class="hljs-string">https://raw.githubusercontent.com/$TARGET_USER/$repo/main/README.md)</span>

              {
                <span class="hljs-string">echo</span> <span class="hljs-string">"---"</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"title: \"$title\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"description: \"$description\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"clone_url: \"$clone_url\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"language: \"$language\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"homepage: \"$homepage\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"topics:"</span>
                <span class="hljs-string">for</span> <span class="hljs-string">topic</span> <span class="hljs-string">in</span> <span class="hljs-string">$(echo</span> <span class="hljs-string">"$topics"</span> <span class="hljs-string">|</span> <span class="hljs-string">tr</span> <span class="hljs-string">','</span> <span class="hljs-string">'\n'</span><span class="hljs-string">);</span> <span class="hljs-string">do</span>
                  <span class="hljs-string">echo</span> <span class="hljs-string">"  - \"$topic\""</span>
                <span class="hljs-string">done</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"created_at: \"$created_at\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"updated_at: \"$updated_at\""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"---"</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">""</span>
                <span class="hljs-string">echo</span> <span class="hljs-string">"$readme_content"</span>
              } <span class="hljs-string">&gt;</span> <span class="hljs-string">"$readme_file"</span>
            <span class="hljs-string">else</span>
              <span class="hljs-string">echo</span> <span class="hljs-string">"No README found for $repo, skipping..."</span>
            <span class="hljs-string">fi</span>
          <span class="hljs-string">done</span>

      <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">Commit</span> <span class="hljs-string">Changes</span>
        <span class="hljs-attr">run:</span> <span class="hljs-string">|
          git config --global user.name "github-actions[bot]"
          git config --global user.email "github-actions[bot]@users.noreply.github.com"
</span>
          <span class="hljs-string">git</span> <span class="hljs-string">add</span> <span class="hljs-string">-A</span>

          <span class="hljs-comment"># Commit if there are any changes</span>
          <span class="hljs-string">if</span> <span class="hljs-string">!</span> <span class="hljs-string">git</span> <span class="hljs-string">diff</span> <span class="hljs-string">--cached</span> <span class="hljs-string">--quiet;</span> <span class="hljs-string">then</span>
            <span class="hljs-string">git</span> <span class="hljs-string">commit</span> <span class="hljs-string">-m</span> <span class="hljs-string">"Update READMEs with meta and content from public repos"</span>
            <span class="hljs-string">git</span> <span class="hljs-string">push</span> <span class="hljs-string">https://x-access-token:${{</span> <span class="hljs-string">secrets.&lt;secret_name&gt;</span> <span class="hljs-string">}}@github.com/&lt;username&gt;/&lt;repository_name&gt;.git</span> <span class="hljs-string">HEAD:&lt;branch_name&gt;</span>
          <span class="hljs-string">else</span>
            <span class="hljs-string">echo</span> <span class="hljs-string">"No changes to commit."</span>
          <span class="hljs-string">fi</span>
</code></pre>
<p>The GitHub Actions workflow runs daily, fetching all the user's repositories, writing their metadata and README content (if any), and saving them as <code>&lt;repository_name&gt;_README.mdx</code> files.</p>
<p>By adding <code>workflow_dispatch:</code> below the cron timer, we enable support for manually triggering the workflow whenever needed.</p>
<p>If you are following along, make sure that you change the placeholder texts with the actual name.</p>
<p>Now, that we have all the metadata as well as repository README contents, we can use any Markdown parsers to parse the <code>mdx</code> content and show it into our site.</p>
<p>If you want to see my projects page in action, visit 👇</p>
<p><a target="_blank" href="https://techwithshrijal.com/projects">https://techwithshrijal.com/projects</a></p>
<hr />
<h2 id="heading-conclusion">Conclusion ⚡</h2>
<p>With this simple GitHub Action, your portfolio site will always be in sync with your GitHub projects. By using basic command-line scripting, we automated the process without relying on third-party libraries.</p>
<p>You can find the entire source code here: <a target="_blank" href="https://github.com/shricodev/portfolio.git">https://github.com/shricodev/portfolio.git</a></p>
<p>Check out my portfolio here: <a target="_blank" href="https://techwithshrijal.com">https://techwithshrijal.com</a></p>
<p>Thank you for reading! 🎉</p>
<p><img src="https://i.giphy.com/media/v1.Y2lkPTc5MGI3NjExOW51YXlzeWh0dzFnampmMmkxbm5tYnZtc2RkanBlamxxbDlmMG10ZCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/YitBbyU7DNjUs/giphy.gif" alt="Leonardo Dicaprio Cheers" /></p>
]]></content:encoded></item><item><title><![CDATA[Build your own personal SIRI with LLAMA-3 like a PRO! 🧙‍♂️ 🪄]]></title><description><![CDATA[In this easy-to-follow tutorial, you will learn how to build your own voice assistant Siri with the LLAMA-3 AI Model. 😎 What you will learn: 👀 Learn how…]]></description><link>https://shricodev.hashnode.dev/build-your-own-personal-siri-with-llama-3-like-a-pro</link><guid isPermaLink="true">https://shricodev.hashnode.dev/build-your-own-personal-siri-with-llama-3-like-a-pro</guid><category><![CDATA[Python]]></category><category><![CDATA[LLaMa]]></category><category><![CDATA[voice assistants]]></category><category><![CDATA[openai]]></category><category><![CDATA[shell script]]></category><category><![CDATA[groq]]></category><category><![CDATA[tts]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Thu, 17 Oct 2024 13:05:05 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1729323317347/7b9dc1a5-9b9e-4941-aec0-2a46fdc2a73c.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR ✨</h2>
<p>In this easy-to-follow tutorial, you will learn how to build your own voice assistant Siri with the LLAMA-3 AI Model. 😎</p>
<p><strong>What you will learn: 👀</strong></p>
<ul>
<li><p>Learn how to set up <strong>TTS</strong> in a Python project using OpenAI TTS / Pyttsx3 / gTTS.</p>
</li>
<li><p>Learn to generate chat response using <strong>Groq</strong> with <strong>LLAMA-3</strong> model.</p>
</li>
<li><p>Learn to capture webcam images and process it using the Google Generative AI.</p>
</li>
<li><p>Learn to automate all the manual tasks with <strong>shell scripting</strong>.</p>
</li>
</ul>
<p>Let's build this thing together! 😵‍💫</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1729320626797/016bed74-cf3b-4199-b88b-e7fa9435d32b.gif" alt="Ready GIF" /></p>
<hr />
<h2 id="heading-setting-up-the-environment">Setting up the Environment 🛠️</h2>
<p>Create a folder to keep all your source code for the project:</p>
<pre><code class="lang-bash">mkdir siri-voice-llama3
<span class="hljs-built_in">cd</span> siri-voice-llama3
</code></pre>
<p>Create a few new subfolders where we will store the source code, shell scripts, logs and chat history:</p>
<pre><code class="lang-bash">mkdir -p src logs src/scripts data/ai_response data/chat_history
</code></pre>
<p>Now that the initial folder structure is set up, it's time to create a new virtual environment and install all the modules we will be using in our project.</p>
<p>Run these commands to create and activate a new virtual environment in the root of our project:</p>
<pre><code class="lang-bash">python3 -m venv .venv
<span class="hljs-built_in">source</span> .venv/bin/activate <span class="hljs-comment"># If you are using fish shell, change the activate binary to activate.fish</span>
</code></pre>
<p>Run this command to install all the necessary modules we will be using in our project:</p>
<pre><code class="lang-bash">pip3 install SpeechRecognition opencv-python openai google-generativeai gTTS pyttsx3 groq faster-whisper numpy python-dotenv pyperclip pydub PyAudio pillow
</code></pre>
<blockquote>
<p>⚠️ <strong>NOTE:</strong> Installing packages this way can lead to issues if they change in the future. For exact versions, please find my <code>requirements.txt</code> file <a target="_blank" href="https://github.com/shricodev/siri-voice-llama3/blob/main/requirements.txt">here</a>. Copy the contents to create this file in your project's root directory.</p>
<p>To install, run: <code>pip3 install -r requirements.txt</code></p>
</blockquote>
<p>Here is what each module is used for:</p>
<ul>
<li><p><code>SpeechRecognition</code>: Enables speech recognition from audio files or streams.</p>
</li>
<li><p><code>opencv-python</code>: Used for processing webcam images.</p>
</li>
<li><p><code>groq</code>: A library for working with Groq. Used to generate response from LLAMA-3.</p>
</li>
<li><p><code>google-generativeai</code>: Used in image processing to provide context.</p>
</li>
<li><p><code>faster-whisper</code>: A faster implementation of Whisper for speech recognition.</p>
</li>
<li><p><code>python-dotenv</code>: For reading key-value pairs from a .env file.</p>
</li>
<li><p><code>pyperclip</code>: Facilitates clipboard operations (copying and pasting).</p>
</li>
<li><p><code>pydub</code>: Handles audio manipulation tasks.</p>
</li>
<li><p><code>PyAudio</code>: Manages audio input/output.</p>
</li>
<li><p><code>numpy</code>: Supports numerical computing and efficient array handling.</p>
</li>
<li><p><code>pillow</code>: A fork of the Python Imaging Library (PIL) for image processing.</p>
</li>
</ul>
<p><strong>Optional Modules:</strong></p>
<blockquote>
<p>ℹ️ Among these, only one is required.</p>
</blockquote>
<ul>
<li><p><code>openai</code>: Enables Text-to-Speech using OpenAI's streaming audio.</p>
</li>
<li><p><code>gTTS</code>: Google Text-to-Speech library for generating speech from text.</p>
</li>
<li><p><code>pyttsx3</code>: A Python Text-to-Speech library for offline speech synthesis.</p>
</li>
</ul>
<hr />
<h2 id="heading-lets-code-it">Let's Code It 💻</h2>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1729320628135/9620e708-4736-4fb1-b92f-c77cab9c1aa8.gif" alt="Fire GIF" /></p>
<h3 id="heading-setting-up-chat-history-support">Setting Up Chat History Support 📋</h3>
<blockquote>
<p>💡 We are going to add support for chat history in a log file for each day seperately.</p>
</blockquote>
<p>Inside the <code>src</code> directory, add a file named <code>utils.py</code> with the following code:</p>
<blockquote>
<p>In this file, we will store all the helper functions that we will require in our program.</p>
</blockquote>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/utils.py</span>

<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> sys
<span class="hljs-keyword">from</span> datetime <span class="hljs-keyword">import</span> datetime
<span class="hljs-keyword">from</span> pathlib <span class="hljs-keyword">import</span> Path
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> Literal, NoReturn, Optional

<span class="hljs-keyword">import</span> pyperclip
<span class="hljs-keyword">from</span> PIL <span class="hljs-keyword">import</span> ImageGrab

<span class="hljs-keyword">import</span> utils


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_log_file_for_today</span>(<span class="hljs-params">project_root_folder_path: Path</span>) -&gt; Path:</span>
    <span class="hljs-string">"""
    Retrieves the log file path for today's date, ensuring that the necessary
    directories are created. If the log file for the current day does not exist,
    it creates an empty log file.

    Args:
        project_root_folder_path (Path): The root folder of the project, where the 'data'
        directory resides.

    Returns:
        Path: The absolute path to the log file for today's date.
    """</span>

    today = datetime.today()

    <span class="hljs-comment"># The year is always 4 digit and the month, day is always 2 digit using this format.</span>
    year = today.strftime(<span class="hljs-string">"%Y"</span>)
    month = today.strftime(<span class="hljs-string">"%m"</span>)
    day = today.strftime(<span class="hljs-string">"%d"</span>)

    base_folder = os.path.join(
        project_root_folder_path, <span class="hljs-string">"data"</span>, <span class="hljs-string">"chat_history"</span>, year, month
    )

    os.makedirs(base_folder, exist_ok=<span class="hljs-literal">True</span>)
    chat_log_file = os.path.join(base_folder, <span class="hljs-string">f"<span class="hljs-subst">{day}</span>.log"</span>)

    Path(chat_log_file).touch(exist_ok=<span class="hljs-literal">True</span>)

    <span class="hljs-keyword">return</span> Path(os.path.abspath(chat_log_file))


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">log_chat_message</span>(<span class="hljs-params">
    log_file_path: Path,
    user_message: Optional[str] = None,
    ai_message: Optional[str] = None,
</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Logs user and assistant chat messages to the provided log file, along with
    a timestamp. Either the user message or the assistant message (or both) can
    be provided.

    Args:
        log_file_path (Path): The absolute path to the log file where messages will be logged.
        user_message (Optional[str]): The message sent by the user. Defaults to None.
        ai_message (Optional[str]): The message generated by the assistant. Defaults to None.

    Returns:
        None: This function appends the messages to the log file in a readable format
        with a timestamp. It does not return anything.
    """</span>

    <span class="hljs-comment"># If neither of the message is given, return.</span>
    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> user_message <span class="hljs-keyword">and</span> <span class="hljs-keyword">not</span> ai_message:
        <span class="hljs-keyword">return</span>

    timestamp = datetime.now().strftime(<span class="hljs-string">"[%H : %M : %S]"</span>)

    <span class="hljs-keyword">with</span> open(log_file_path, <span class="hljs-string">"a"</span>) <span class="hljs-keyword">as</span> log_file:
        <span class="hljs-keyword">if</span> user_message:
            log_file.write(<span class="hljs-string">f"<span class="hljs-subst">{timestamp}</span> - USER: <span class="hljs-subst">{user_message}</span>"</span>)

        <span class="hljs-keyword">if</span> ai_message:
            log_file.write(<span class="hljs-string">f"<span class="hljs-subst">{timestamp}</span> - ASSISTANT: <span class="hljs-subst">{ai_message}</span>\n"</span>)

        log_file.write(<span class="hljs-string">"\n"</span>)
</code></pre>
<p>The <code>get_log_file_for_today</code> function takes in a path to the project root folder, which is usually where our <code>main.py</code> file will be located.</p>
<p>It constructs a path to today’s log file stored in <code>data/chat_history/{month}/{day}.log</code>. If the file doesn't exist, it creates an empty file and returns the path. If it does exist, it simply returns the existing path.</p>
<p>The <code>log_chat_message</code> function takes the path to the log file, the user message, and the AI message, then logs the received messages with a specific timestamp.</p>
<hr />
<h3 id="heading-api-keys-configuration">API Keys Configuration 🔑</h3>
<p>For this project, we will require a few API keys. That includes Groq key, Google Generative AI key and optionally OpenAI key.</p>
<p>Create a new file <code>.env</code> in the root of the project and populate it with the API keys.</p>
<pre><code class="lang-bash"><span class="hljs-comment"># Required</span>
GROQ_API_KEY=
GOOGLE_GENERATIVE_AI_API_KEY=

<span class="hljs-comment"># Optional</span>
OPENAI_API_KEY=
</code></pre>
<p>After populating the <code>.env</code> file with the API keys, it's time to get access to it in our Python code.</p>
<p>Inside the <code>src</code> directory, create a new file <code>setup.py</code> with the following code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/setup.py</span>

<span class="hljs-keyword">import</span> os

<span class="hljs-keyword">from</span> dotenv <span class="hljs-keyword">import</span> load_dotenv

<span class="hljs-keyword">import</span> utils


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_credentials</span>() -&gt; tuple[str, str, str | <span class="hljs-keyword">None</span>]:</span>
    <span class="hljs-string">"""
    Load API keys from environment variables and return them as a tuple.

    This function loads environment variables from a `.env` file using `dotenv`.
    It retrieves the Groq API key, Google Generative AI API key, and OpenAI API key.
    If any of the keys are missing, it exits the program with an error message.

    Returns:
        tuple[str, str, str | None]: A tuple containing the Groq API key, Google Generative AI API key,
                              and OpenAI API key.

    Raises:
        SystemExit: If any of the required API keys are not found, the program exits with an error message.
    """</span>
    load_dotenv()

    groq_api_key: str | <span class="hljs-literal">None</span> = os.getenv(<span class="hljs-string">"GROQ_API_KEY"</span>)
    google_gen_ai_api_key: str | <span class="hljs-literal">None</span> = os.getenv(<span class="hljs-string">"GOOGLE_GENERATIVE_AI_API_KEY"</span>)
    openai_api_key: str | <span class="hljs-literal">None</span> = os.getenv(<span class="hljs-string">"OPENAI_API_KEY"</span>)

    <span class="hljs-keyword">if</span> groq_api_key <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span> <span class="hljs-keyword">or</span> google_gen_ai_api_key <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span>:
        <span class="hljs-keyword">return</span> utils.exit_program(
            status_code=<span class="hljs-number">1</span>,
            message=<span class="hljs-string">"Missing required API key(s). Make sure to set them in `.env` file. If you are using the OpenAI approach, then populate the OpenAI api key as well."</span>,
        )

    <span class="hljs-keyword">return</span> groq_api_key, google_gen_ai_api_key, openai_api_key
</code></pre>
<p>The <code>get_credentials</code> function loads API keys from environment variables using the <code>dotenv</code> library and returns them as a tuple.</p>
<p>If either the Groq or Google API keys are missing, the function exits the program with an error message, prompting the user to set the necessary keys in a <code>.env</code> file. The OpenAI key is returned as optional and can be None if not set.</p>
<hr />
<h3 id="heading-defining-additional-helper-functions">Defining Additional Helper Functions 👷</h3>
<p>Above in the <code>get_credentials</code> function in <code>setup.py</code>, we are using <code>utils.exit_program</code>, but we have not defined it yet.</p>
<p>Let's work on that and add some more helper functions that we will need in the project.</p>
<p>Inside the <code>utils.py</code> file in the <code>src</code> directory, add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/utils.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">exit_program</span>(<span class="hljs-params">status_code: int = <span class="hljs-number">0</span>, message: str = <span class="hljs-string">""</span></span>) -&gt; NoReturn:</span>
    <span class="hljs-string">"""
    Exit the program with an optional error message.

    Args:
        status_code (int): The exit status code. Defaults to 0 (success).
        message (str): An optional error message to display before exiting.
    """</span>

    <span class="hljs-keyword">if</span> message:
        print(<span class="hljs-string">f"ERROR: <span class="hljs-subst">{message}</span>\n"</span>)
    sys.exit(status_code)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_path_to_folder</span>(<span class="hljs-params">folder_type: Literal[<span class="hljs-string">"webcam"</span>, <span class="hljs-string">"screenshot"</span>]</span>) -&gt; Path:</span>
    <span class="hljs-string">"""
    Get the path to the specified folder type (webcam or screenshot).

    Args:
        folder_type (Literal["webcam", "screenshot"]): The type of folder to retrieve the path for.

    Returns:
        Path: The path to the specified folder.

    Raises:
        ValueError: If the folder_type is not valid.
    """</span>

    base_path = Path(os.path.join(Path.home(), <span class="hljs-string">"Pictures"</span>, <span class="hljs-string">"llama3.1"</span>))
    folder_map = {
        <span class="hljs-string">"screenshot"</span>: Path(os.path.join(base_path, <span class="hljs-string">"Screenshots"</span>)),
        <span class="hljs-string">"webcam"</span>: Path(os.path.join(base_path, <span class="hljs-string">"Webcam"</span>)),
    }

    <span class="hljs-keyword">if</span> folder_type <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> folder_map:
        <span class="hljs-keyword">raise</span> ValueError(
            <span class="hljs-string">f"ERROR: Invalid folder_type: <span class="hljs-subst">{folder_type}</span>. Expected 'webcam' or 'screenshot'."</span>
        )

    <span class="hljs-keyword">return</span> folder_map[folder_type]
</code></pre>
<p>The <code>exit_program</code> as the name suggests it exits the program with a specified status code and optional error message. If a message is provided, it prints it before exiting.</p>
<p>The <code>get_path_to_folder</code> function constructs and returns the folder path for a specified folder type, either "webcam" or "screenshot." It combines the user's home directory with a predefined base path ("Pictures/llama3.1") and appends the relevant folder name. We will use this function to store images in the appropriate folder, either for webcam or screenshots.</p>
<p>Now, we will define few more functions that deals with capturing and removing screenshots and getting clipboard text.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/utils.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">capture_screenshot</span>() -&gt; Path:</span>
    <span class="hljs-string">"""
    Captures a screenshot and saves it to the designated folder.

    Returns:
        Path: The file path of the saved screenshot.
    """</span>

    screenshot_folder_path = utils.get_path_to_folder(folder_type=<span class="hljs-string">"screenshot"</span>)

    os.makedirs(screenshot_folder_path, exist_ok=<span class="hljs-literal">True</span>)

    screen = ImageGrab.grab()

    time_stamp = datetime.now().strftime(<span class="hljs-string">"%Y_%m_%d_%H_%M_%S"</span>)
    rgb_screenshot = screen.convert(<span class="hljs-string">"RGB"</span>)

    image_filename = <span class="hljs-string">f"screenshot_<span class="hljs-subst">{time_stamp}</span>.png"</span>
    image_file_path = Path(os.path.join(screenshot_folder_path, image_filename))

    rgb_screenshot.save(image_file_path, quality=<span class="hljs-number">20</span>)

    <span class="hljs-keyword">return</span> image_file_path


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">remove_last_screenshot</span>() -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Remove the most recent screenshot file from the designated screenshots folder.

    The function checks if the folder exists and if there are any .png files. If
    found, it deletes the most recently created screenshot.
    """</span>

    folder_path = utils.get_path_to_folder(folder_type=<span class="hljs-string">"screenshot"</span>)

    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(folder_path):
        <span class="hljs-keyword">return</span>

    files = [
        file
        <span class="hljs-keyword">for</span> file <span class="hljs-keyword">in</span> os.listdir(folder_path)
        <span class="hljs-keyword">if</span> os.path.isfile(os.path.join(folder_path, file)) <span class="hljs-keyword">and</span> file.endswith(<span class="hljs-string">".png"</span>)
    ]
    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> files:
        <span class="hljs-keyword">return</span>

    most_recent_file = max(
        files, key=<span class="hljs-keyword">lambda</span> f: os.path.getctime(os.path.join(folder_path, f))
    )

    os.remove(os.path.join(folder_path, most_recent_file))


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_clipboard_text</span>() -&gt; str:</span>
    <span class="hljs-string">"""
    Retrieves the current text content from the system clipboard.

    This function uses the `pyperclip` module to access the clipboard. If the clipboard
    content is a valid string, it returns the content. If the content is not a string,
    it returns an empty string.

    Returns:
        str: The text content from the clipboard, or an empty string if the content is
        not a string or the clipboard is empty.
    """</span>

    clipboard_content = pyperclip.paste()

    <span class="hljs-keyword">if</span> isinstance(clipboard_content, str):
        <span class="hljs-keyword">return</span> clipboard_content

    <span class="hljs-keyword">return</span> <span class="hljs-string">""</span>
</code></pre>
<p>The <code>capture_screenshot</code> function captures the current screen using the <strong>ImageGrab</strong> module, saves it as a PNG file in a designated screenshots folder, and returns the full file path of the saved screenshot. 📸</p>
<p>It constructs the filename with a timestamp to ensure uniqueness and sets a quality of 20 for the saved image. We are degrading the quality of the image so to make it fast to process the image later.</p>
<p>The <code>remove_last_screenshot</code> function identifies and deletes the most recently created screenshot from the designated folder. It first checks if the folder exists and looks for .png files within it. If files are found, it uses the creation time to determine the most recent file before removing it. 🚮</p>
<p>The <code>get_clipboard_text</code> function uses the <code>pyperclip</code> module to access the clipboard. If the clipboard content is a valid string, it returns the content. If the content is not a string, it returns an empty string.</p>
<hr />
<h3 id="heading-integrating-webcam-support">Integrating Webcam Support 📸</h3>
<p>For capturing images from the webcam, we will have to add support to it.</p>
<p>Inside the <code>src</code> directory, create a new file <code>webcam.py</code> and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/webcam.py</span>

<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">from</span> datetime <span class="hljs-keyword">import</span> datetime
<span class="hljs-keyword">from</span> pathlib <span class="hljs-keyword">import</span> Path
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> NoReturn, Union

<span class="hljs-keyword">import</span> cv2

<span class="hljs-keyword">import</span> utils


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_available_webcam</span>() -&gt; cv2.VideoCapture | <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Checks for available webcams and returns the first one that is opened.

    This function attempts to open the first 10 webcam indices. If a webcam is found
    and successfully opened, it returns a VideoCapture object. If no webcams are found,
    it exits the program with an error message.

    Returns:
        cv2.VideoCapture: The opened webcam object.
        None: If no webcam is found, the program exits with an error message.
    """</span>

    <span class="hljs-comment"># Assuming that we are checking the first 10 webcams.</span>
    <span class="hljs-keyword">for</span> index <span class="hljs-keyword">in</span> range(<span class="hljs-number">10</span>):
        web_cam = cv2.VideoCapture(index)
        <span class="hljs-keyword">if</span> web_cam.isOpened():
            <span class="hljs-keyword">return</span> web_cam

    <span class="hljs-keyword">return</span> utils.exit_program(status_code=<span class="hljs-number">1</span>, message=<span class="hljs-string">"No webcams found."</span>)


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">capture_webcam_image</span>() -&gt; Union[Path, NoReturn]:</span>
    <span class="hljs-string">"""
    Captures an image from the available webcam and saves it to the specified folder.

    This function first checks for an available webcam using `get_available_webcam`.
    If a webcam is successfully opened, it creates a folder for saving the images if
    it does not already exist, generates a timestamped filename, captures a frame,
    and saves the image to the specified folder. The function then releases the webcam.

    Returns:
        Path: The file path of the saved image.
        NoReturn: If there was an error capturing the image, the program exits with an error message.
    """</span>

    webcam = get_available_webcam()
    <span class="hljs-keyword">if</span> webcam <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span> <span class="hljs-keyword">or</span> <span class="hljs-keyword">not</span> webcam.isOpened():
        <span class="hljs-keyword">return</span> utils.exit_program(
            status_code=<span class="hljs-number">1</span>, message=<span class="hljs-string">"There was an error capturing the image."</span>
        )

    webcam_folder_path = utils.get_path_to_folder(folder_type=<span class="hljs-string">"webcam"</span>)

    os.makedirs(webcam_folder_path, exist_ok=<span class="hljs-literal">True</span>)

    timestamp = datetime.now().strftime(<span class="hljs-string">"%Y_%m_%d_%H_%M_%S"</span>)
    image_filename = <span class="hljs-string">f"webcam_<span class="hljs-subst">{timestamp}</span>.png"</span>

    _, frame = webcam.read()

    image_file_path_str = os.path.join(webcam_folder_path, image_filename)

    cv2.imwrite(image_file_path_str, frame)

    webcam.release()

    <span class="hljs-keyword">return</span> Path(image_file_path_str)
</code></pre>
<p>The <code>get_available_webcam</code> function checks for available webcams, assuming the first ten indices. If a webcam is successfully opened, it returns the corresponding <code>cv2.VideoCapture</code> object. 🎥 If no webcams are found, it exits the program with an error message.</p>
<p>The <code>capture_webcam_image</code> function captures an image from the available webcam. It first calls our previously written helper function <code>get_available_webcam</code> to get an available webcam. If successful, it creates a folder for saving images (if it doesn’t already exist), generates a timestamped filename, captures a frame, and saves the image. Finally, it releases the webcam and returns the path of the saved image. 🖼️</p>
<hr />
<h3 id="heading-implementing-the-main-program-logic">Implementing the Main Program Logic 😵‍💫</h3>
<p>Now that we have coded all the utilities that we will be requiring when working on the project, Let's start with the main program logic</p>
<p>Inside the <code>src</code> directory, create a new file <code>siri.py</code> and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/siri.py</span>

<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> re
<span class="hljs-keyword">import</span> time
<span class="hljs-keyword">from</span> pathlib <span class="hljs-keyword">import</span> Path
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> List

<span class="hljs-keyword">import</span> google.generativeai <span class="hljs-keyword">as</span> genai
<span class="hljs-keyword">import</span> pyttsx3
<span class="hljs-keyword">import</span> speech_recognition <span class="hljs-keyword">as</span> sr
<span class="hljs-keyword">from</span> faster_whisper <span class="hljs-keyword">import</span> WhisperModel
<span class="hljs-keyword">from</span> groq <span class="hljs-keyword">import</span> Groq
<span class="hljs-keyword">from</span> groq.types.chat <span class="hljs-keyword">import</span> ChatCompletionMessageParam
<span class="hljs-keyword">from</span> gtts <span class="hljs-keyword">import</span> gTTS
<span class="hljs-keyword">from</span> openai <span class="hljs-keyword">import</span> OpenAI
<span class="hljs-keyword">from</span> PIL <span class="hljs-keyword">import</span> Image
<span class="hljs-keyword">from</span> pydub <span class="hljs-keyword">import</span> AudioSegment
<span class="hljs-keyword">from</span> pydub.playback <span class="hljs-keyword">import</span> play

<span class="hljs-keyword">import</span> utils
<span class="hljs-keyword">import</span> webcam


<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Siri</span>:</span>
    <span class="hljs-string">"""
    A multi-modal AI voice assistant that responds to user prompts
    by processing voice commands and context from images or clipboard content.
    """</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">
        self,
        log_file_path: Path,
        project_root_folder_path: Path,
        groq_api_key: str,
        google_gen_ai_api_key: str,
        openai_api_key: str | None,
    </span>) -&gt; <span class="hljs-keyword">None</span>:</span>
        <span class="hljs-string">"""
        Initializes the Siri assistant with API clients for Groq, OpenAI, and Google Generative AI.

        Args:
            log_file_path (Path): Path to the log file.
            project_root_folder_path (Path): Root folder of the project.
            groq_api_key (str): API key for Groq.
            google_gen_ai_api_key (str): API key for Google Generative AI.
            openai_api_key (str): API key for OpenAI.
        """</span>
        self.log_file_path = log_file_path
        self.project_root_folder_path = project_root_folder_path

        self.pyttsx3_engine = pyttsx3.init()

        self.groq_client = Groq(api_key=groq_api_key)
        self.openai_client = OpenAI(api_key=openai_api_key)

        <span class="hljs-comment"># Configure Google Generative AI model</span>
        genai_generation_config = genai.GenerationConfig(
            temperature=<span class="hljs-number">0.7</span>, top_p=<span class="hljs-number">1</span>, top_k=<span class="hljs-number">1</span>, max_output_tokens=<span class="hljs-number">2048</span>
        )
        genai.configure(api_key=google_gen_ai_api_key)

        self.genai_model = genai.GenerativeModel(
            <span class="hljs-string">"gemini-1.5-flash-latest"</span>,
            generation_config=genai_generation_config,
            safety_settings=[
                {<span class="hljs-string">"category"</span>: <span class="hljs-string">"HARM_CATEGORY_HARASSMENT"</span>, <span class="hljs-string">"threshold"</span>: <span class="hljs-string">"BLOCK_NONE"</span>},
                {<span class="hljs-string">"category"</span>: <span class="hljs-string">"HARM_CATEGORY_HATE_SPEECH"</span>, <span class="hljs-string">"threshold"</span>: <span class="hljs-string">"BLOCK_NONE"</span>},
                {
                    <span class="hljs-string">"category"</span>: <span class="hljs-string">"HARM_CATEGORY_SEXUALLY_EXPLICIT"</span>,
                    <span class="hljs-string">"threshold"</span>: <span class="hljs-string">"BLOCK_NONE"</span>,
                },
                {
                    <span class="hljs-string">"category"</span>: <span class="hljs-string">"HARM_CATEGORY_DANGEROUS_CONTENT"</span>,
                    <span class="hljs-string">"threshold"</span>: <span class="hljs-string">"BLOCK_NONE"</span>,
                },
            ],
        )

        <span class="hljs-comment"># Initialize conversation context for the AI</span>
        self.conversation: List[ChatCompletionMessageParam] = [
            {
                <span class="hljs-string">"role"</span>: <span class="hljs-string">"user"</span>,
                <span class="hljs-string">"content"</span>: (
                    <span class="hljs-string">"You are a multi-modal AI voice assistant. Your user may have attached a photo (screenshot or webcam capture) "</span>
                    <span class="hljs-string">"for context, which has already been processed into a detailed text prompt. This will be attached to their transcribed "</span>
                    <span class="hljs-string">"voice input. Generate the most relevant and factual response by carefully considering all previously generated text "</span>
                    <span class="hljs-string">"before adding new information. Do not expect or request additional images; use the provided context if available. "</span>
                    <span class="hljs-string">"Please do not include newlines in your response. Keep it all in one paragraph. "</span>
                    <span class="hljs-string">"Ensure your responses are clear, concise, and relevant to the ongoing conversation, avoiding any unnecessary verbosity."</span>
                ),
            }
        ]

        total_cpu_cores = os.cpu_count() <span class="hljs-keyword">or</span> <span class="hljs-number">1</span>

        <span class="hljs-comment"># Initialize the audio transcription model</span>
        self.audio_transcription_model = WhisperModel(
            device=<span class="hljs-string">"cpu"</span>,
            compute_type=<span class="hljs-string">"int8"</span>,
            model_size_or_path=<span class="hljs-string">"base"</span>,
            cpu_threads=total_cpu_cores // <span class="hljs-number">2</span>,
            num_workers=total_cpu_cores // <span class="hljs-number">2</span>,
        )

        <span class="hljs-comment"># Initialize speech recognition components</span>
        self.speech_recognizer = sr.Recognizer()
        self.mic_audio_source = sr.Microphone()
        self.wake_word = <span class="hljs-string">"siri"</span>
</code></pre>
<p>The <strong>Siri</strong> class sets up a multi-modal AI voice assistant that not only handles voice commands but also processes images or clipboard content to add context.</p>
<p>It takes in a few key parameters, like the <code>log_file_path</code> and <code>project_root_folder_path</code> help with logging conversations and storing AI responses as mp3 files when using gTTS. You’ll also need API keys for Groq, Google Generative AI, and optionally OpenAI. 🤖</p>
<p>The class sets up clients for Groq, OpenAI, and Google Generative AI. For Google GenAI, it uses the <strong>gemini-1.5-flash</strong> model and tweaks some of the safety settings.</p>
<p>An initial conversation prompt is built in to guide the AI on how to respond based on the user’s voice commands or any processed images. 💬</p>
<p>For audio transcription, it uses the Faster Whisper model, which runs on the CPU with specific settings for performance. It also sets up speech recognition using <code>Recognizer</code> and <code>Microphone</code>, with the assistant listening for the wake word <strong>"siri"</strong> to start taking commands.</p>
<p>Now that the initial configuration is done, let's add some methods to the class. We will define a few more to handle audio transcription, extract the user's prompt, listen for the prompt, and generate chat responses using Groq. 🛠️</p>
<p>Add the following methods in the <code>siri.py</code> file inside the <code>src</code> directory.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/siri.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">transcribe_audio_to_text</span>(<span class="hljs-params">self, audio_file_path: Path</span>) -&gt; str:</span>
    <span class="hljs-string">"""
    Transcribes audio from a file to text.

    Args:
        audio_file_path (Path): Path to the audio file.

    Returns:
        str: The transcribed text from the audio.
    """</span>

    segments, _ = self.audio_transcription_model.transcribe(
        audio=str(audio_file_path)
    )
    <span class="hljs-keyword">return</span> <span class="hljs-string">""</span>.join(segment.text <span class="hljs-keyword">for</span> segment <span class="hljs-keyword">in</span> segments)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">extract_prompt</span>(<span class="hljs-params">self, transcribed_text: str</span>) -&gt; str | <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Extracts the user's prompt from the transcribed text after the wake word.

    Args:
        transcribed_text (str): The transcribed text from audio input.

    Returns:
        str | None: The extracted prompt if found, otherwise None.
    """</span>

    pattern = <span class="hljs-string">rf"\b<span class="hljs-subst">{re.escape(self.wake_word)}</span>[\s,.?!]*([A-Za-z0-9].*)"</span>
    regex_match = re.search(
        pattern=pattern, string=transcribed_text, flags=re.IGNORECASE
    )

    <span class="hljs-keyword">if</span> regex_match <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span>:
        <span class="hljs-keyword">return</span> <span class="hljs-literal">None</span>

    <span class="hljs-keyword">return</span> regex_match.group(<span class="hljs-number">1</span>).strip()

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">listen</span>(<span class="hljs-params">self</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Starts listening for the wake word and processes audio input in the background.
    """</span>

    <span class="hljs-keyword">with</span> self.mic_audio_source <span class="hljs-keyword">as</span> mic:
        self.speech_recognizer.adjust_for_ambient_noise(source=mic, duration=<span class="hljs-number">2</span>)

    self.speech_recognizer.listen_in_background(
        source=self.mic_audio_source, callback=self.handle_audio_processing
    )

    <span class="hljs-keyword">while</span> <span class="hljs-literal">True</span>:
        time.sleep(<span class="hljs-number">0.5</span>)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">generate_chat_response_with_groq</span>(<span class="hljs-params">
    self, prompt: str, image_context: str | None
</span>) -&gt; str:</span>
    <span class="hljs-string">"""
    Generates a response from the Groq model based on user input and optional image context.

    Args:
        prompt (str): The user's prompt.
        image_context (str | None): Optional image context for the response.

    Returns:
        str: The generated response from the assistant.
    """</span>

    <span class="hljs-keyword">if</span> image_context:
        prompt = <span class="hljs-string">f"USER_PROMPT: <span class="hljs-subst">{prompt}</span>\n\nIMAGE_CONTEXT: <span class="hljs-subst">{image_context}</span>"</span>

    self.conversation.append({<span class="hljs-string">"role"</span>: <span class="hljs-string">"user"</span>, <span class="hljs-string">"content"</span>: prompt})

    completion = self.groq_client.chat.completions.create(
        messages=self.conversation, model=<span class="hljs-string">"llama-3.1-8b-instant"</span>
    )

    ai_response = completion.choices[<span class="hljs-number">0</span>].message.content

    self.conversation.append({<span class="hljs-string">"role"</span>: <span class="hljs-string">"assistant"</span>, <span class="hljs-string">"content"</span>: ai_response})

    <span class="hljs-keyword">return</span> ai_response <span class="hljs-keyword">or</span> <span class="hljs-string">"Sorry, I'm not sure how to respond to that."</span>
</code></pre>
<p>The method <code>transcribe_audio_to_text</code> takes the path to an audio file and transcribes its content into text. It uses the WhisperModel to process the audio file in segments and returns a string that concatenates the transcribed text from all segments. 🎧</p>
<p>The method <code>extract_prompt</code> extracts the user’s spoken prompt from the transcribed text, specifically after the wake word (e.g., "siri"). It uses a regular expression to find and capture the prompt following the wake word, returning the cleaned-up prompt or <code>None</code> if no prompt is found. 🗣️</p>
<p>The method <code>listen</code> continuously listens for the wake word and processes audio input. It first adjusts for ambient noise, then starts listening in the background using a callback (<code>handle_audio_processing</code>). The method enters an infinite loop, pausing briefly in each iteration to keep listening. 🔄</p>
<p>The method <code>generate_chat_response_with_groq</code> generates a response using the Groq model based on the user’s prompt and optional image context. It formats the prompt with the image context (if any), adds the conversation to the model, and appends the AI’s response to the ongoing conversation. It then returns the generated response or a default message if no response is generated. 💬</p>
<hr />
<h4 id="heading-text-to-speech-generation">Text-to-Speech Generation 🗣️</h4>
<p>For Text To Speech Generation we will be implementing three different approaches. pyttsx3, OpenAI and gTTS (Google Text To Speech). You are free to choose any approach that fits your requirement.</p>
<ul>
<li><strong>Pyttsx3 Approach</strong></li>
</ul>
<p>Here for text to speech generation, we will be using a famous Python Module Pyttsx3.</p>
<p>In the <code>siri.py</code> file, add the following method.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/siri.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-comment"># Pyttsx3 Approach (Weaker Audio Quality)</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">text_to_speech</span>(<span class="hljs-params">self, text: str</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Converts text to speech using Pyttsx3's text-to-speech API.

    Args:
        text (str): The text to convert to speech.
    """</span>

    self.pyttsx3_engine.setProperty(<span class="hljs-string">"volume"</span>, <span class="hljs-number">1.0</span>)
    self.pyttsx3_engine.setProperty(<span class="hljs-string">"rate"</span>, <span class="hljs-number">125</span>)

    voices = self.pyttsx3_engine.getProperty(<span class="hljs-string">"voices"</span>)

    <span class="hljs-comment"># Set voice to Female.</span>
    self.pyttsx3_engine.setProperty(<span class="hljs-string">"voice"</span>, voices[<span class="hljs-number">0</span>].id)

    self.pyttsx3_engine.say(text)
    self.pyttsx3_engine.runAndWait()

    self.pyttsx3_engine.stop()
</code></pre>
<p>The <code>text_to_speech</code> method uses the <code>pyttsx3_engine</code> we initialized inside the Siri class to set some of the properties for the engine and finally say the text that we provide to it.</p>
<ul>
<li><strong>OpenAI Approach</strong></li>
</ul>
<p>For this approach, we will be using the OpenAI Audio speech with streaming. This approach has overall the best experience than any other approaches, but it requires you to have OpenAI API setup and your account having some OpenAI credits.</p>
<p>In the <code>siri.py</code> file, add the following method.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/siri.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-comment"># OpenAI Approach (Best Quality Audio with multiple voice available).</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">text_to_speech</span>(<span class="hljs-params">self, text: str</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Converts text to speech using OpenAI's text-to-speech API.

    Args:
        text (str): The text to convert to speech.
    """</span>

    stream = pyaudio.PyAudio().open(
        format=pyaudio.paInt16, channels=<span class="hljs-number">1</span>, rate=<span class="hljs-number">24000</span>, output=<span class="hljs-literal">True</span>
    )
    stream_start = <span class="hljs-literal">False</span>

    <span class="hljs-keyword">with</span> self.openai_client.audio.speech.with_streaming_response.create(
        model=<span class="hljs-string">"tts-1"</span>, voice=<span class="hljs-string">"nova"</span>, response_format=<span class="hljs-string">"pcm"</span>, input=text
    ) <span class="hljs-keyword">as</span> openai_response:
        silence_threshold = <span class="hljs-number">0.1</span>
        <span class="hljs-keyword">for</span> chunk <span class="hljs-keyword">in</span> openai_response.iter_bytes(chunk_size=<span class="hljs-number">1024</span>):
            <span class="hljs-keyword">if</span> stream_start:
                stream.write(chunk)

            <span class="hljs-keyword">elif</span> max(chunk) &gt; silence_threshold:
                stream.write(chunk)
                stream_start = <span class="hljs-literal">True</span>
</code></pre>
<p>The <code>text_to_speech</code> method converts text into speech using OpenAI's text-to-speech (TTS) API.</p>
<p>It starts by opening an audio stream using <code>PyAudio</code>, which is configured to output audio at a sample rate of 24,000 Hz with 16-bit resolution. The method then calls the OpenAI TTS API with the provided text, specifying the model ("tts-1"), voice ("nova"), and response format ("pcm"). The audio data is streamed in real time. 🚀</p>
<p>You can change the voices if you like. For a list of available options, visit <a target="_blank" href="https://platform.openai.com/docs/guides/text-to-speech/overview">here</a>.</p>
<p>Within the loop, the method checks the audio chunks returned by the OpenAI API. If the audio exceeds a certain silence threshold, the stream begins playing the audio chunks, ensuring the text is spoken only when meaningful sound is detected. This prevents the stream from starting with silence. 🔊</p>
<ul>
<li><strong>gTTS Approach</strong></li>
</ul>
<p>For this approach, we will be using the Google Text to Speech Engine. This approach is pretty slow and also there's a need to save the AIs response as an 'mp3' and then play that audio file.</p>
<p>In the <code>siri.py</code> file, add the following method.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/siri.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">text_to_speech</span>(<span class="hljs-params">self, text: str</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Converts text to speech using Google's text-to-speech API.

    Args:
        text (str): The text to convert to speech.
    """</span>

    tts = gTTS(text=text, lang=<span class="hljs-string">"en"</span>, slow=<span class="hljs-literal">False</span>)

    response_folder_path = Path(
        os.path.abspath(
            os.path.join(self.project_root_folder_path, <span class="hljs-string">"data"</span>, <span class="hljs-string">"ai_response"</span>)
        )
    )

    os.makedirs(response_folder_path, exist_ok=<span class="hljs-literal">True</span>)

    response_audio_file_path = Path(
        os.path.join(response_folder_path, <span class="hljs-string">"ai_response_audio.mp3"</span>)
    )

    tts.save(response_audio_file_path)

    response_audio = AudioSegment.from_mp3(response_audio_file_path)
    play(response_audio)

    <span class="hljs-comment"># After the audio is played, delete the audio file.</span>
    <span class="hljs-keyword">if</span> os.path.exists(response_audio_file_path):
        os.remove(response_audio_file_path)
</code></pre>
<p>The <code>text_to_speech</code> method converts text to speech using Google's TTS API. It first generates speech from the given text in English, setting <code>slow=False</code> to speed up the playback. The method then creates a folder path for storing the response audio file inside a "data/ai_response" directory. After ensuring the directory exists, it saves the speech as an mp3 file.</p>
<p>Once the mp3 file is saved, it loads the audio using <code>AudioSegment</code> and plays it. After playing the audio, the method deletes the mp3 file to clean up.</p>
<hr />
<p>Now, that we've also worked on the <code>text_to_speech</code> method, we need to write a few more methods that deals with analyzing image prompt if the user attaches image context to the prompt, select the relevant assistant action and process the audio and take relevant actions.</p>
<p>Add the following code to the <code>siri.py</code> file in the <code>src</code> directory.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/src/siri.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">analyze_image_prompt</span>(<span class="hljs-params">self, prompt: str, image_path: Path</span>) -&gt; str:</span>
    <span class="hljs-string">"""
    Analyzes an image based on the user prompt to extract semantic information.

    Args:
        prompt (str): The user's prompt related to the image.
        image_path (Path): The path to the image file.

    Returns:
        str: The analysis result from the image based on the prompt.
    """</span>

    image = Image.open(image_path)
    prompt = (
        <span class="hljs-string">"You are an image analysis AI tasked with extracting semantic meaning from images to assist another AI in "</span>
        <span class="hljs-string">"generating a user response. Your role is to analyze the image based on the user's prompt and provide all relevant, "</span>
        <span class="hljs-string">"objective data without directly responding to the user. Focus solely on interpreting the image in the context of "</span>
        <span class="hljs-string">f"the user’s request and relay that information for further processing. \nUSER_PROMPT: <span class="hljs-subst">{prompt}</span>"</span>
    )
    genai_response = self.genai_model.generate_content([prompt, image])
    <span class="hljs-keyword">return</span> genai_response.text


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">select_assistant_action</span>(<span class="hljs-params">self, prompt: str</span>) -&gt; str:</span>
    <span class="hljs-string">"""
    Determines the appropriate action for the assistant to take based on user input.

    Args:
        prompt (str): The user's prompt.

    Returns:
        str: The selected action for the assistant.
    """</span>

    system_prompt_message = (
        <span class="hljs-string">"You are an AI model tasked with selecting the most appropriate action for a voice assistant. Based on the user's prompt, "</span>
        <span class="hljs-string">"choose one of the following actions: ['extract clipboard', 'take screenshot', 'delete screenshot', 'capture webcam', 'generic']. "</span>
        <span class="hljs-string">"Assume the webcam is a standard laptop webcam facing the user. Provide only the action without explanations or additional text. "</span>
        <span class="hljs-string">"Respond strictly with the most suitable option from the list."</span>
    )
    function_conversation: List[ChatCompletionMessageParam] = [
        {<span class="hljs-string">"role"</span>: <span class="hljs-string">"system"</span>, <span class="hljs-string">"content"</span>: system_prompt_message},
        {<span class="hljs-string">"role"</span>: <span class="hljs-string">"user"</span>, <span class="hljs-string">"content"</span>: prompt},
    ]

    completion = self.groq_client.chat.completions.create(
        messages=function_conversation, model=<span class="hljs-string">"llama-3.1-8b-instant"</span>
    )

    ai_response = completion.choices[<span class="hljs-number">0</span>].message.content

    <span class="hljs-keyword">return</span> ai_response <span class="hljs-keyword">or</span> <span class="hljs-string">"generic"</span>


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">handle_audio_processing</span>(<span class="hljs-params">self, recognizer: sr.Recognizer, audio: sr.AudioData</span>):</span>
    <span class="hljs-string">"""
    Callback function to process audio input once recognized.

    Args:
        recognizer (sr.Recognizer): The speech recognizer instance.
        audio (sr.AudioData): The audio data captured by the microphone.
    """</span>

    data_folder_path = Path(os.path.abspath(os.path.join(<span class="hljs-string">"."</span>, <span class="hljs-string">"data"</span>)))
    os.makedirs(data_folder_path, exist_ok=<span class="hljs-literal">True</span>)

    audio_prompt_file_path = Path(
        os.path.abspath(os.path.join(data_folder_path, <span class="hljs-string">"user_audio_prompt.wav"</span>))
    )
    <span class="hljs-keyword">with</span> open(audio_prompt_file_path, <span class="hljs-string">"wb"</span>) <span class="hljs-keyword">as</span> f:
        f.write(audio.get_wav_data())

    transcribed_text = self.transcribe_audio_to_text(
        audio_file_path=audio_prompt_file_path
    )
    parsed_prompt = self.extract_prompt(transcribed_text=transcribed_text)

    <span class="hljs-keyword">if</span> parsed_prompt:
        utils.log_chat_message(
            log_file_path=self.log_file_path, user_message=parsed_prompt
        )
        skip_response = <span class="hljs-literal">False</span>

        selected_assistant_action = self.select_assistant_action(
            prompt=parsed_prompt
        )

        <span class="hljs-keyword">if</span> <span class="hljs-string">"capture webcam"</span> <span class="hljs-keyword">in</span> selected_assistant_action:
            image_path = webcam.capture_webcam_image()
            image_analysis_result = self.analyze_image_prompt(
                prompt=parsed_prompt, image_path=image_path
            )

        <span class="hljs-keyword">elif</span> <span class="hljs-string">"take screenshot"</span> <span class="hljs-keyword">in</span> selected_assistant_action:
            image_path = utils.capture_screenshot()
            image_analysis_result = self.analyze_image_prompt(
                prompt=parsed_prompt, image_path=image_path
            )

        <span class="hljs-keyword">elif</span> <span class="hljs-string">"delete screenshot"</span> <span class="hljs-keyword">in</span> selected_assistant_action:
            utils.remove_last_screenshot()
            image_analysis_result = <span class="hljs-literal">None</span>
            ai_response = <span class="hljs-string">"Screenshot deleted successfully."</span>
            self.text_to_speech(text=ai_response)

            utils.log_chat_message(
                log_file_path=self.log_file_path, ai_message=ai_response
            )

            skip_response = <span class="hljs-literal">True</span>

        <span class="hljs-keyword">elif</span> <span class="hljs-string">"extract clipboard"</span> <span class="hljs-keyword">in</span> selected_assistant_action:
            clipboard_content = utils.get_clipboard_text()
            parsed_prompt = (
                <span class="hljs-string">f"<span class="hljs-subst">{parsed_prompt}</span>\n\nCLIPBOARD_CONTENT: <span class="hljs-subst">{clipboard_content}</span>"</span>
            )
            image_analysis_result = <span class="hljs-literal">None</span>

        <span class="hljs-keyword">else</span>:
            image_analysis_result = <span class="hljs-literal">None</span>

        <span class="hljs-comment"># If the response is not supposed to be skipped, then generate the response and speak it out.</span>
        <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> skip_response:
            response = self.generate_chat_response_with_groq(
                prompt=parsed_prompt, image_context=image_analysis_result
            )
            utils.log_chat_message(
                log_file_path=self.log_file_path, ai_message=response
            )
            self.text_to_speech(text=response)

    <span class="hljs-comment"># Remove the user prompt audio after the response is generated.</span>
    <span class="hljs-keyword">if</span> os.path.exists(audio_prompt_file_path):
        os.remove(audio_prompt_file_path)
</code></pre>
<p>The <code>analyze_image_prompt</code> method analyzes an image based on a user’s prompt to extract semantic information. It starts by opening the specified image file using the PIL library. The method then constructs a prompt that instructs the image analysis AI to focus on extracting relevant data from the image without responding directly to the user.</p>
<p>The method sends the constructed prompt and the image to the Google Generative AI model for processing. Finally, it returns the image analysis result as text. 📝</p>
<p>The <code>select_assistant_action</code> method determines the appropriate action for the assistant based on the user's input. 🤔 It starts by creating a system prompt that instructs the AI model to choose from a predefined list of actions: <code>'extract clipboard', 'take screenshot', 'delete screenshot', 'capture webcam', or 'generic'</code>.</p>
<p>Next, the method constructs a conversation list that includes the system prompt and the user’s prompt. It then sends this conversation to the Groq client to generate a response using the specified model. The response can be any one of the item from the predefined list of actions.</p>
<p>The <code>handle_audio_processing</code> method processes audio input after it’s recognized by the assistant. First, it saves the captured audio as a <code>.wav</code> file in the "data" folder. It then transcribes the audio to text using the <code>transcribe_audio_to_text</code> method and extracts the user’s prompt from the text with <code>extract_prompt</code>.</p>
<p>If a prompt is found, it logs the user’s message and determines the appropriate assistant action using <code>select_assistant_action</code>. Depending on the action, it may capture a webcam image, take a screenshot, delete a screenshot, or extract clipboard content. For image-based actions, it uses <code>analyze_image_prompt</code> to analyze the image. 🔍</p>
<p>The <code>skip_response</code> variable is used to control whether the assistant should skip generating and speaking a response after performing certain actions. It is initially set to False, meaning the response generation is expected.</p>
<p>For example, when the action is "delete screenshot," the method deletes the screenshot and directly provides a pre-defined response ("Screenshot deleted successfully.") via text-to-speech. In this case, <code>skip_response</code> is set to True to prevent the assistant from generating a separate response for the user prompt, as the action itself is sufficient. ✅</p>
<p>For other actions, it generates a response using <code>generate_chat_response_with_groq</code> method and converts the response to speech. After the response is generated, the method deletes the audio file. 🚮</p>
<h3 id="heading-writing-the-mainpy-file">Writing the <code>main.py</code> File 🧑‍💻</h3>
<p>This is going to be the entry point for our program. It performs the setup and initialization needed for the assistant to function.</p>
<p>Create a new file called <code>main.py</code> inside the root of the project and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 siri-voice-llama3/main.py</span>

<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> sys
<span class="hljs-keyword">from</span> pathlib <span class="hljs-keyword">import</span> Path

<span class="hljs-comment"># Add the src directory to the module search path</span>
sys.path.insert(<span class="hljs-number">0</span>, os.path.join(os.path.dirname(os.path.abspath(__file__)), <span class="hljs-string">"src"</span>))

<span class="hljs-keyword">from</span> src <span class="hljs-keyword">import</span> setup, siri, utils

<span class="hljs-string">"""
Main entry point for the AI llama3 siri voice assitant.

This script loads the necessary API credentials from environment variables,
initializes the Siri assistant with the provided keys, and starts listening
for user input. The program will exit if any of the required API keys are
missing.

To run the application, execute this script in an environment where the
`.env` file is properly configured with the required API keys.
"""</span>

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    <span class="hljs-comment"># Determine the current directory of the script</span>
    project_root_folder_path = Path(os.path.dirname(os.path.abspath(__file__)))

    chat_log_file_path = utils.get_log_file_for_today(
        project_root_folder_path=project_root_folder_path
    )

    all_api_keys = setup.get_credentials()
    groq_api_key, google_gen_ai_api_key, openai_api_key = all_api_keys

    siri = siri.Siri(
        log_file_path=chat_log_file_path,
        project_root_folder_path=project_root_folder_path,
        groq_api_key=groq_api_key,
        google_gen_ai_api_key=google_gen_ai_api_key,
        openai_api_key=openai_api_key,
    )

    siri.listen()
</code></pre>
<blockquote>
<p>👀 Notice that we are inserting the path to our src directory using the <code>sys.path.insert()</code> method to ensure that Python can locate and import modules from the <code>src</code> directory.</p>
</blockquote>
<p>The main block starts by determining the project root folder, then gets the daily log file path using <code>utils.get_log_file_for_today</code> to log the chat messages.</p>
<p>Next, we retrieve APIs (for Groq, Google Generative AI, and OpenAI) using <code>setup.get_credentials</code> function that we wrote earlier when working on coding some helper functions.</p>
<p>We then create an instance of the Siri class, passing the log file path, project root folder path, and API keys.</p>
<p>Finally, the <code>siri.listen</code> method is called, which starts the assistant and listens for user input.</p>
<p>By now, you should have a working version of your own Voice Assitant. 🥂</p>
<hr />
<h2 id="heading-optional-building-a-shell-script">Optional: Building a Shell Script 🧰</h2>
<p>🤔 <strong>Why is there a need to write a shell script?</strong></p>
<blockquote>
<p>Well, there is no need to. I actually wrote this shell script myself, thinking that I could automate running it on system reboot through scheduling tools like Linux services or cron jobs. However, I couldn't get it to work since it requires access to hardware components (like the mic), so it didn’t really speak out the response (Do let me know if you find a fix). But this shell script can come in really handy if you want to automate every manual steps, like creating a virtual environment, installing dependencies, and finally running the program. You could also add the script to your <strong>PATH</strong> via symlink and run it from anywhere on your system. 😉</p>
</blockquote>
<p>Create a new file <code>start_siri_llama3.sh</code> inside the <code>src/scripts</code> directory with the following lines of code:</p>
<blockquote>
<p>💁 If you are using the fish shell, you can find the same code with the fish syntax <a target="_blank" href="https://github.com/shricodev/siri-voice-llama3/blob/main/src/scripts/start_siri_llama3.fish">here</a>. Create a new file called <code>start_siri_llama3.fish</code> inside the <code>src/scripts</code> directory, and add the code from the link.</p>
</blockquote>
<pre><code class="lang-bash"><span class="hljs-comment"># 👇 siri-voice-llama3/src/scripts/start_siri_llama3.sh</span>

<span class="hljs-comment">#!/usr/bin/env bash</span>
<span class="hljs-comment"># Using this above way of writing shebang can have some security concerns.</span>
<span class="hljs-comment"># See this stackoverflow thread: https://stackoverflow.com/a/72332845</span>
<span class="hljs-comment"># Since, I want this script to be portable for most of the users, instead of hardcoding like '#!/usr/bin/bash', I am using this way.</span>

ERROR_USAGE=<span class="hljs-string">"ERROR: Usage: bash {path_to_main.py}"</span>
ERROR_FILE_NOT_FOUND=<span class="hljs-string">"ERROR: The main.py file does not exist or is not a valid file."</span>
ERROR_PYTHON_NOT_FOUND=<span class="hljs-string">"ERROR: No suitable Python executable found."</span>
ERROR_BASH_NOT_INSTALLED=<span class="hljs-string">"ERROR: Bash shell is not installed. Please install Bash."</span>
ERROR_ACTIVATE_NOT_FOUND=<span class="hljs-string">"ERROR: activate file not found in '<span class="hljs-variable">$VENV_DIR</span>/bin'"</span>
ERROR_UNSUPPORTED_SHELL=<span class="hljs-string">"ERROR: Unsupported shell: '<span class="hljs-variable">$SHELL</span>'"</span>
ERROR_REQUIREMENTS_NOT_FOUND=<span class="hljs-string">"ERROR: requirements.txt file not found in '<span class="hljs-variable">$SCRIPT_DIR</span>'"</span>

<span class="hljs-comment"># Determine the script directory, virtual environment directory, and log file</span>
SCRIPT_DIR=<span class="hljs-string">"<span class="hljs-subst">$(dirname <span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$0</span>"</span>)</span>"</span>)</span>"</span>
VENV_DIR=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$SCRIPT_DIR</span>/../../.venv"</span>)</span>"</span>
LOG_FILE=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$SCRIPT_DIR</span>/../../logs/shell-error-bash.log"</span>)</span>"</span>
REQUIREMENTS_FILE_PATH=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$SCRIPT_DIR</span>/../../requirements.txt"</span>)</span>"</span>

<span class="hljs-function"><span class="hljs-title">log_and_exit</span></span>() {
  <span class="hljs-built_in">local</span> message=<span class="hljs-string">"<span class="hljs-variable">$1</span>"</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"[<span class="hljs-subst">$(date +<span class="hljs-string">"%Y-%m-%d %H:%M:%S"</span>)</span>] <span class="hljs-variable">$message</span>"</span> | tee -a <span class="hljs-variable">$LOG_FILE</span>
  <span class="hljs-built_in">exit</span> 1
}

<span class="hljs-comment"># Check if the main.py file is provided as an argument</span>
<span class="hljs-keyword">if</span> [ <span class="hljs-variable">$#</span> -ne 1 ]; <span class="hljs-keyword">then</span>
  log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_USAGE</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Function to check if a file exists and has the correct extension</span>
<span class="hljs-function"><span class="hljs-title">check_file</span></span>() {
    <span class="hljs-built_in">local</span> file_path=<span class="hljs-string">"<span class="hljs-variable">$1</span>"</span>
    <span class="hljs-built_in">local</span> expected_extension=<span class="hljs-string">"<span class="hljs-variable">$2</span>"</span>

    <span class="hljs-keyword">if</span> [ ! -f <span class="hljs-string">"<span class="hljs-variable">$file_path</span>"</span> ]; <span class="hljs-keyword">then</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_FILE_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>

    <span class="hljs-keyword">if</span> ! [[ <span class="hljs-string">"<span class="hljs-variable">$file_path</span>"</span> == *<span class="hljs-string">".<span class="hljs-variable">$expected_extension</span>"</span> ]]; <span class="hljs-keyword">then</span>
        log_and_exit <span class="hljs-string">"The file '<span class="hljs-variable">$file_path</span>' must be a '.<span class="hljs-variable">$expected_extension</span>' file."</span>
    <span class="hljs-keyword">fi</span>
}

<span class="hljs-comment"># Validate the provided main.py file</span>
check_file <span class="hljs-string">"<span class="hljs-variable">$1</span>"</span> <span class="hljs-string">"py"</span>

<span class="hljs-comment"># Extract and validate arguments</span>
MAIN_FILE_PATH=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$1</span>"</span>)</span>"</span>

<span class="hljs-comment"># Find the appropriate Python executable</span>
PYTHON_EXEC=<span class="hljs-string">"<span class="hljs-subst">$(command -v python3 || command -v python)</span>"</span>

<span class="hljs-comment"># Ensure that the Python executable is available before creating the virtual environment</span>
<span class="hljs-keyword">if</span> [ ! -d <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>"</span> ]; <span class="hljs-keyword">then</span>
    <span class="hljs-keyword">if</span> [ -z <span class="hljs-string">"<span class="hljs-variable">$PYTHON_EXEC</span>"</span> ]; <span class="hljs-keyword">then</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_PYTHON_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>

    <span class="hljs-string">"<span class="hljs-variable">$PYTHON_EXEC</span>"</span> -m venv <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>"</span>

    <span class="hljs-comment"># Activate the virtual environment after creating it</span>
    <span class="hljs-keyword">if</span> [ -f <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>/bin/activate"</span> ]; <span class="hljs-keyword">then</span>
        <span class="hljs-built_in">source</span> <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>/bin/activate"</span>
    <span class="hljs-keyword">else</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_ACTIVATE_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>

    PIP_EXEC_VENV = <span class="hljs-string">"<span class="hljs-subst">$(command -v pip3 || command -v pip)</span>"</span>

    <span class="hljs-comment"># Check if requirements.txt exists and install dependencies</span>
    <span class="hljs-keyword">if</span> [ -f <span class="hljs-string">"<span class="hljs-variable">$REQUIREMENTS_FILE_PATH</span>"</span> ]; <span class="hljs-keyword">then</span>
        <span class="hljs-string">"<span class="hljs-variable">$PIP_EXEC_VENV</span>"</span> install -r <span class="hljs-string">"<span class="hljs-variable">$REQUIREMENTS_FILE_PATH</span>"</span>
    <span class="hljs-keyword">else</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_REQUIREMENTS_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Ensure that the Bash shell is installed.</span>
<span class="hljs-keyword">if</span> ! <span class="hljs-built_in">command</span> -v bash &amp;&gt; /dev/null; <span class="hljs-keyword">then</span>
    log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_BASH_NOT_INSTALLED</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Activate the virtual environment based on the shell type</span>
<span class="hljs-keyword">if</span> [[ <span class="hljs-string">"<span class="hljs-variable">$SHELL</span>"</span> == *<span class="hljs-string">"/bash"</span> ]]; <span class="hljs-keyword">then</span>
    <span class="hljs-comment"># Check if the activate file exists before sourcing it</span>
    <span class="hljs-keyword">if</span> [ -f <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>/bin/activate"</span> ]; <span class="hljs-keyword">then</span>
        <span class="hljs-built_in">source</span> <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>/bin/activate"</span>
    <span class="hljs-keyword">else</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_ACTIVATE_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>
<span class="hljs-keyword">else</span>
    log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_UNSUPPORTED_SHELL</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Set the python executable to the one from the virtual environment</span>
PYTHON_EXEC=<span class="hljs-string">"<span class="hljs-subst">$(command -v python3 || command -v python)</span>"</span>

<span class="hljs-comment"># Run the main.py file</span>
<span class="hljs-string">"<span class="hljs-variable">$PYTHON_EXEC</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$MAIN_FILE_PATH</span>"</span>
</code></pre>
<p>This script is designed to automate the setup and execution of a Python program, ensuring the necessary environment is prepared before running <code>main.py</code>. ⚙️</p>
<p>First, it checks whether a valid Python file (<code>main.py</code>) is passed as an argument. If not, it logs an error and exits. It also verifies the file exists and has the correct extension (<code>.py</code>). 🐍</p>
<p>The script then searches for a Python executable (<code>python3</code> or <code>python</code>), and if a virtual environment (venv) doesn't exist, it creates one using Python's <code>venv</code> module. Once the venv is created, it activates it, and installs dependencies from <code>requirements.txt</code> if found. The script ensures that both Python and Bash are installed on the system, as it only supports the Bash shell.</p>
<p>If the user's shell isn't Bash, it logs an error and exits. Otherwise, it runs the Python script (<code>main.py</code>) inside the virtual environment using the Python executable found.</p>
<p>Now, to actually be able to run this script from anywhere on your system, you can add it to your <strong>PATH</strong> with symlink. 🔗</p>
<p>Usually, <code>/usr/local/bin</code> is a place where we add our custom built scripts. Firstly make sure that it is in your <strong>PATH</strong>, by running the command:</p>
<pre><code class="lang-bash"><span class="hljs-built_in">echo</span> <span class="hljs-variable">$PATH</span>
</code></pre>
<p>If not add it to your <strong>PATH</strong>, then you can add this script as a symlink to <code>/usr/local/bin</code> with the following command:</p>
<pre><code class="lang-bash">ln -s {absolute_path_to_script_sh/fish} /usr/<span class="hljs-built_in">local</span>/bin/start_siri_llama3
</code></pre>
<p>After running this command, you should now be able to run this program from anywhere on your system. 🎉</p>
<hr />
<h2 id="heading-conclusion">Conclusion ⚡</h2>
<p>Wow! 😮‍💨 We've done a lot together! If you've made it this far, give yourself a well-deserved pat on the back. By now, you’ve successfully built a personal SIRI voice assistant using the LLAMA-3 AI model.</p>
<p>The entire documented source code for this article is available here:</p>
<p>https://github.com/shricodev/siri-voice-llama3.git</p>
<p>Thank you so much for reading! 🎉 🫡</p>
<blockquote>
<p>Drop down your thoughts in the comment section below. 👇</p>
</blockquote>
<p>Follow me on Socials 🐥: https://linktr.ee/shricodev</p>
]]></content:encoded></item><item><title><![CDATA[Learn to monitor your Python Application Like a PRO! 🧙‍♂️🪄]]></title><description><![CDATA[In this easy-to-follow tutorial, you'll discover how to keep an eye on your Python application using distributed tracing. What you will learn: ✨ How to…]]></description><link>https://shricodev.hashnode.dev/learn-to-monitor-your-python-application-like-a-pro</link><guid isPermaLink="true">https://shricodev.hashnode.dev/learn-to-monitor-your-python-application-like-a-pro</guid><category><![CDATA[Python]]></category><category><![CDATA[Docker]]></category><category><![CDATA[Kubernetes]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Programming Blogs]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Fri, 16 Aug 2024 14:15:13 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1723815380491/08a7a287-b670-45d0-8711-c3ce6018f011.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR</h2>
<p>In this easy-to-follow tutorial, you'll discover how to keep an eye on your Python application using <strong>distributed tracing</strong>.</p>
<p><strong>What you will learn: ✨</strong></p>
<ul>
<li><p>How to build microservices in Python 🐍.</p>
</li>
<li><p>Setting up Docker containers 📦 for microservices.</p>
</li>
<li><p>Configuring Kubernetes for managing microservices.</p>
</li>
<li><p>Integrating a tracing backend for visualizing the traces 🕵️‍♂️.</p>
</li>
</ul>
<p>Ready to light up your Python application monitoring skills? 🔥</p>
<p><img src="https://media.giphy.com/media/3o72FfM5HJydzafgUE/giphy.gif" alt="Fire GIF" /></p>
<hr />
<h2 id="heading-setup-instructions">Setup Instructions</h2>
<blockquote>
<p>🚨 In this section of the blog, we'll be building a dummy Python microservices application. If you already have one and are following along, feel free to skip this part.</p>
</blockquote>
<p>Create the initial folder structure for your application as shown below. 👇</p>
<pre><code class="lang-bash">mkdir python-microservices &amp;&amp; <span class="hljs-built_in">cd</span> python-microservices
mkdir src &amp;&amp; <span class="hljs-built_in">cd</span> src
mkdir microservice1 microservice2
</code></pre>
<h3 id="heading-setting-up-the-server">Setting Up the Server 🖥️</h3>
<blockquote>
<p>For demonstration purposes, I will create two microservices that will communicate with each other, and eventually, we can use that to visualize distributed tracing.</p>
</blockquote>
<h4 id="heading-building-and-dockerizing-microservice-1">Building and Dockerizing Microservice 1</h4>
<p>Inside the <code>/microservice1</code> directory, create a new Python virtual environment, install the necessary dependencies, and initialize a Flask application.</p>
<blockquote>
<p>🚨 I assume you are following along on a Unix-based machine. If you are on a Windows machine, some commands will differ.</p>
</blockquote>
<pre><code class="lang-bash"><span class="hljs-built_in">cd</span> microservice1
python -m venv .venv
<span class="hljs-built_in">source</span> .venv/bin/activate
</code></pre>
<blockquote>
<p>💡 If you are on a fish shell, run the below command to activate the virtual environment.</p>
</blockquote>
<pre><code class="lang-bash"><span class="hljs-built_in">source</span> .venv/bin/activate.fish
</code></pre>
<p>Install the required dependencies:</p>
<pre><code class="lang-bash">pip install Flask requests
</code></pre>
<p>Get the list of installed dependencies in the <code>requirements.txt</code> so that we can later use it in our container to install the dependencies.</p>
<pre><code class="lang-bash">pip freeze &gt; requirements.txt
</code></pre>
<p>Create a new file called <a target="_blank" href="http://app.py"><code>app.py</code></a> and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 src/microservice1/app.py</span>
<span class="hljs-keyword">import</span> socket
<span class="hljs-keyword">import</span> requests
<span class="hljs-keyword">from</span> flask <span class="hljs-keyword">import</span> Flask, jsonify, render_template

app = Flask(__name__)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">user_os_details</span>():</span>
    hostname = socket.gethostname()
    hostip = socket.gethostbyname(hostname)
    <span class="hljs-keyword">return</span> hostname, hostip

<span class="hljs-meta">@app.route("/")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">index</span>():</span>
    <span class="hljs-keyword">return</span> <span class="hljs-string">"&lt;p&gt;Welcome to Flask microservice 1&lt;/p&gt;"</span>

<span class="hljs-meta">@app.route("/health")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">health</span>():</span>
    <span class="hljs-keyword">return</span> jsonify(status=<span class="hljs-string">"Microservice 1 Running..."</span>)

<span class="hljs-meta">@app.route("/get-users")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_users</span>():</span>
    response = requests.get(<span class="hljs-string">"http://microservice2:5001/get-gh-users"</span>)
    <span class="hljs-keyword">return</span> render_template(<span class="hljs-string">"index.html"</span>, users=response.json())

<span class="hljs-meta">@app.route("/os-details")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">details</span>():</span>
    host_name, host_ip = user_os_details()
    <span class="hljs-keyword">return</span> jsonify(hostname=host_name, hostip=host_ip)

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    app.run(<span class="hljs-string">"0.0.0.0"</span>, <span class="hljs-number">5000</span>)
</code></pre>
<blockquote>
<p>💡 If you've noticed, we're requesting data from <a target="_blank" href="http://microservice2:5001/get-gh-users"><code>http://microservice2:5001/get-gh-users</code></a>. You might be wondering, what is this microservice2? Well, we can use service names as host names within the same network in docker. We will build this service later once we finish writing and dockerizing this microservice.</p>
</blockquote>
<p>As you can see, this is a very simple Flask application with a few endpoints. The <code>user_os_details()</code> function gets the <strong>Hostname</strong> and <strong>IP address</strong> of the machine.</p>
<p>The <code>@app.route("/")</code> and <code>@app.route("/health")</code> decorators define the root and "/health" endpoints of the Flask app. The "/health" endpoint will be used later to check the container health ❤️‍🩹 in the <code>Dockerfile</code>.</p>
<p>The <code>@app.route("/get-users")</code> and <code>@app.route("/os-details")</code> decorators define the "/get-users" and "/os-details" endpoints. The "/get-users" endpoint fetches GitHub users from <strong>microservice2</strong> and passes them as props to the <code>index.html</code> file for rendering. Meanwhile, the "/os-details" endpoint displays system details. Finally, the application is run on port 5000.</p>
<p>Now, let's create the <code>index.html</code> file where we are rendering the received users from microservice2.</p>
<p>Create a new folder <code>/templates</code> and add an <code>index.html</code> file with the following contents:</p>
<pre><code class="lang-html"><span class="hljs-comment">&lt;!-- 👇 src/microservice1/templates/index.html --&gt;</span>

<span class="hljs-meta">&lt;!DOCTYPE <span class="hljs-meta-keyword">html</span>&gt;</span>
<span class="hljs-tag">&lt;<span class="hljs-name">html</span> <span class="hljs-attr">lang</span>=<span class="hljs-string">"en"</span>&gt;</span>
<span class="hljs-tag">&lt;<span class="hljs-name">head</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">meta</span> <span class="hljs-attr">charset</span>=<span class="hljs-string">"UTF-8"</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">meta</span> <span class="hljs-attr">name</span>=<span class="hljs-string">"viewport"</span> <span class="hljs-attr">content</span>=<span class="hljs-string">"width=device-width, initial-scale=1.0"</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">title</span>&gt;</span>Microservice 1<span class="hljs-tag">&lt;/<span class="hljs-name">title</span>&gt;</span>
<span class="hljs-tag">&lt;/<span class="hljs-name">head</span>&gt;</span>
<span class="hljs-tag">&lt;<span class="hljs-name">body</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">h1</span>&gt;</span>Microservice 1<span class="hljs-tag">&lt;/<span class="hljs-name">h1</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">h2</span>&gt;</span>This is the data received from microservice2:<span class="hljs-tag">&lt;/<span class="hljs-name">h2</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">p</span>&gt;</span>{{ users }}<span class="hljs-tag">&lt;/<span class="hljs-name">p</span>&gt;</span>
<span class="hljs-tag">&lt;/<span class="hljs-name">body</span>&gt;</span>
<span class="hljs-tag">&lt;/<span class="hljs-name">html</span>&gt;</span>
</code></pre>
<p>Our first microservice is all ready. Let's dockerize it. Create a <code>Dockerfile</code> inside the <code>/microservice1</code> directory and add the following lines of code:</p>
<blockquote>
<p>🚨 Make sure to name it exactly <code>Dockerfile</code> with no extensions.</p>
</blockquote>
<pre><code class="lang-Dockerfile"><span class="hljs-comment"># 👇 src/microservice1/Dockerfile</span>
<span class="hljs-comment"># Use Python alpine as the base image.</span>
<span class="hljs-keyword">FROM</span> python:<span class="hljs-number">3.12</span>.<span class="hljs-number">1</span>-alpine3.<span class="hljs-number">18</span>
<span class="hljs-comment"># Optional: Upgrade pip to the latest version.</span>
<span class="hljs-keyword">RUN</span><span class="bash"> pip install --upgrade pip</span>
<span class="hljs-comment"># Create a new user with fewer permissions.</span>
<span class="hljs-keyword">RUN</span><span class="bash"> adduser -D lowkey</span>
<span class="hljs-comment"># Switch to user lowkey</span>
<span class="hljs-keyword">USER</span> lowkey
<span class="hljs-comment"># Change the working directory to ~/app</span>
<span class="hljs-keyword">WORKDIR</span><span class="bash"> /home/lowkey/app</span>
<span class="hljs-comment"># Copy the requirements.txt file required to install the dependencies.</span>
<span class="hljs-keyword">COPY</span><span class="bash"> --chown=lowkey:lowkey requirements.txt requirements.txt</span>
<span class="hljs-comment"># Install the dependencies</span>
<span class="hljs-keyword">RUN</span><span class="bash"> pip install -r requirements.txt</span>
<span class="hljs-comment"># Copy the rest of the files to the current directory in the docker container.</span>
<span class="hljs-keyword">COPY</span><span class="bash"> --chown=lowkey:lowkey . .</span>
<span class="hljs-comment"># Expose port 5000</span>
<span class="hljs-keyword">EXPOSE</span> <span class="hljs-number">5000</span>
<span class="hljs-comment"># Switch to the root user just for installing curl. It is required.</span>
<span class="hljs-keyword">USER</span> root
<span class="hljs-comment"># Install curl. Alpine uses apk as its package manager.</span>
<span class="hljs-keyword">RUN</span><span class="bash"> apk --no-cache add curl</span>
<span class="hljs-comment"># Switch back to user lowkey</span>
<span class="hljs-keyword">USER</span> lowkey
<span class="hljs-comment"># Check the health of the container. The "/health" endpoint is used </span>
<span class="hljs-comment"># to verify the container is up and running.</span>
<span class="hljs-keyword">HEALTHCHECK</span><span class="bash"> --interval=30s --timeout=30s --start-period=30s --retries=5 \ </span>
            <span class="hljs-keyword">CMD</span><span class="bash"> curl -f http://localhost:5000/health || <span class="hljs-built_in">exit</span> 1</span>
<span class="hljs-comment"># Finally, start the application.</span>
<span class="hljs-keyword">ENTRYPOINT</span><span class="bash"> [ <span class="hljs-string">"python"</span>, <span class="hljs-string">"app.py"</span> ]</span>
</code></pre>
<p>Create a <code>.dockerignore</code> file with the names of the files we don't want to push to the container.</p>
<pre><code class="lang-plaintext">__pycache__
.venv
README.md
Dockerfile
.dockerignore
</code></pre>
<p>Now, that is the entire setup for our first microservice. ✨</p>
<h4 id="heading-building-and-dockerizing-microservice-2">Building and Dockerizing Microservice 2</h4>
<p>We will have a setup similar to <strong>microservice1</strong>, with just a few changes here and there.</p>
<p>Do the exact initial setup as we did for the <strong>microservice1</strong>. After that, in the <code>/microservice2</code> folder, create an <a target="_blank" href="http://app.py"><code>app.py</code></a> file and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 src/microservice2/app.py</span>
<span class="hljs-keyword">import</span> random
<span class="hljs-keyword">import</span> requests
<span class="hljs-keyword">from</span> flask <span class="hljs-keyword">import</span> Flask, jsonify, render_template

app = Flask(__name__)

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_gh_users</span>():</span>
    url = <span class="hljs-string">"https://api.github.com/users?per_page=5"</span>

    <span class="hljs-comment"># Choose a random timeout between 1 and 5 seconds</span>
    timeout = random.randint(<span class="hljs-number">3</span>, <span class="hljs-number">6</span>)

    <span class="hljs-keyword">try</span>:
        response = requests.get(url, timeout=timeout)
        <span class="hljs-keyword">return</span> response.json()
    <span class="hljs-keyword">except</span> requests.exceptions.Timeout:
        <span class="hljs-keyword">return</span> {<span class="hljs-string">"error"</span>: <span class="hljs-string">"Request timed out after {} seconds"</span>.format(timeout)}
    <span class="hljs-keyword">except</span> requests.exceptions.RequestException <span class="hljs-keyword">as</span> e:
        <span class="hljs-keyword">return</span> {<span class="hljs-string">"error"</span>: <span class="hljs-string">"Request failed: {}"</span>.format(e)}

<span class="hljs-meta">@app.route("/")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">index</span>():</span>
    <span class="hljs-keyword">return</span> <span class="hljs-string">"&lt;p&gt;Welcome to Flask microservice 2&lt;/p&gt;"</span>

<span class="hljs-meta">@app.route("/get-gh-users")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_users</span>():</span>
    results = []

    <span class="hljs-comment"># Loop through the number of requests and append the results to the list</span>
    <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(<span class="hljs-number">3</span>):
        result = get_gh_users()
        results.append(result)

    <span class="hljs-comment"># Return the list of results as a JSON response</span>
    <span class="hljs-keyword">return</span> jsonify(results)

<span class="hljs-meta">@app.route("/health")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">health</span>():</span>
    <span class="hljs-keyword">return</span> jsonify(status=<span class="hljs-string">"Microservice 2 Running..."</span>)

<span class="hljs-meta">@app.route("/os-details")</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">details</span>():</span>
    <span class="hljs-keyword">try</span>:
        response = requests.get(<span class="hljs-string">"http://microservice1:5000/os-details"</span>).json()
        host_name = response[<span class="hljs-string">"hostname"</span>]
        host_ip = response[<span class="hljs-string">"hostip"</span>]
        <span class="hljs-keyword">return</span> render_template(<span class="hljs-string">"index.html"</span>, hostname=host_name, hostip=host_ip)
    <span class="hljs-keyword">except</span> requests.exceptions.Timeout <span class="hljs-keyword">as</span> errt:
        <span class="hljs-keyword">return</span> {<span class="hljs-string">"error"</span>: <span class="hljs-string">"Request timed out after {} seconds"</span>.format(errt)}
    <span class="hljs-keyword">except</span> requests.exceptions.RequestException <span class="hljs-keyword">as</span> e:
        <span class="hljs-keyword">return</span> {<span class="hljs-string">"error"</span>: <span class="hljs-string">"Request failed: {}"</span>.format(e)}

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    app.run(<span class="hljs-string">"0.0.0.0"</span>, <span class="hljs-number">5001</span>)
</code></pre>
<p>The <code>@app.route("/")</code> decorator defines the root endpoint of the Flask app, returning a welcome message. The <code>@app.route("/health")</code> decorator defines the "/health" endpoint, which can be used to check the health status of the container.</p>
<p>The <code>@app.route("/get-gh-users")</code> decorator defines the "/get-gh-users" endpoint, which uses the <code>get_gh_users()</code> function to fetch GitHub users and return them as a JSON response. Lastly, the <code>@app.route("/os-details")</code> decorator defines the "/os-details" endpoint, which retrieves operating system details from <strong>microservice1</strong> and renders them in the <code>index.html</code> file. Finally, the application runs on port 5001.</p>
<p>Now, let's create the <code>index.html</code> file where we are rendering the received users from <strong>microservice2</strong>.</p>
<p>Create a new folder <code>/templates</code> and add an <code>index.html</code> file with the following contents:</p>
<pre><code class="lang-html"><span class="hljs-comment">&lt;!-- 👇 src/microservice2/templates/index.html --&gt;</span>

<span class="hljs-meta">&lt;!DOCTYPE <span class="hljs-meta-keyword">html</span>&gt;</span>
<span class="hljs-tag">&lt;<span class="hljs-name">html</span> <span class="hljs-attr">lang</span>=<span class="hljs-string">"en"</span>&gt;</span>
<span class="hljs-tag">&lt;<span class="hljs-name">head</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">meta</span> <span class="hljs-attr">charset</span>=<span class="hljs-string">"UTF-8"</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">meta</span> <span class="hljs-attr">name</span>=<span class="hljs-string">"viewport"</span> <span class="hljs-attr">content</span>=<span class="hljs-string">"width=device-width, initial-scale=1.0"</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">title</span>&gt;</span>Microservice 2<span class="hljs-tag">&lt;/<span class="hljs-name">title</span>&gt;</span>
<span class="hljs-tag">&lt;/<span class="hljs-name">head</span>&gt;</span>
<span class="hljs-tag">&lt;<span class="hljs-name">body</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">h1</span>&gt;</span>Microservice 2<span class="hljs-tag">&lt;/<span class="hljs-name">h1</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">h2</span>&gt;</span>This is the hostname and IP address received from the microservice1:<span class="hljs-tag">&lt;/<span class="hljs-name">h2</span>&gt;</span>
  <span class="hljs-tag">&lt;<span class="hljs-name">p</span>&gt;</span>{{ hostname }} - {{ hostip }}<span class="hljs-tag">&lt;/<span class="hljs-name">p</span>&gt;</span>
<span class="hljs-tag">&lt;/<span class="hljs-name">body</span>&gt;</span>
<span class="hljs-tag">&lt;/<span class="hljs-name">html</span>&gt;</span>
</code></pre>
<p>Now, it's time to dockerize this microservice as well. Copy and paste the entire <code>Dockerfile</code> content for <strong>microservice1</strong> and just change the port from 5000 to 5001.</p>
<p>Also, add a <code>.dockerignore</code> file and include the same files that we added when creating <strong>microservice1</strong>.</p>
<p>Now, that is the entire setup for our second microservice as well. ✨</p>
<hr />
<h4 id="heading-building-dockerfiles-with-docker-compose">Building Dockerfiles with Docker Compose</h4>
<blockquote>
<p>We will follow the best practices of image building by using Docker Compose instead of building each image manually. Here we have only two images, but imagine if we had hundreds or thousands of Dockerfiles. Building each manually would be a tedious task. 😴</p>
</blockquote>
<p>In the root of the project, create a new file named <code>docker-compose.yaml</code> and add the following code:</p>
<pre><code class="lang-yaml"><span class="hljs-attr">services:</span>
  <span class="hljs-attr">microservice1:</span>
    <span class="hljs-attr">build:</span>
      <span class="hljs-attr">context:</span> <span class="hljs-string">./src/microservice1</span>
      <span class="hljs-attr">dockerfile:</span> <span class="hljs-string">Dockerfile</span>
    <span class="hljs-attr">image:</span> <span class="hljs-string">microservice1-image:1.0</span>
    <span class="hljs-attr">ports:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-string">"5000:5000"</span>
    <span class="hljs-attr">restart:</span> <span class="hljs-string">always</span>

  <span class="hljs-attr">microservice2:</span>
    <span class="hljs-attr">build:</span>
      <span class="hljs-attr">context:</span> <span class="hljs-string">./src/microservice2</span>
      <span class="hljs-attr">dockerfile:</span> <span class="hljs-string">Dockerfile</span>
    <span class="hljs-attr">image:</span> <span class="hljs-string">microservice2-image:1.0</span>
    <span class="hljs-attr">ports:</span>
      <span class="hljs-bullet">-</span> <span class="hljs-string">"5001:5001"</span>
    <span class="hljs-attr">restart:</span> <span class="hljs-string">always</span>
</code></pre>
<p>This Docker Compose file defines two services, <strong>microservice1</strong> and <strong>microservice2</strong>. Each service is built using its respective <code>Dockerfile</code> located in the <code>/src</code> directory, with <strong>microservice1</strong> mapped to port 5000 and <strong>microservice2</strong> to port 5001.</p>
<p>The resulting images are tagged <code>microservice1-image:1.0</code> and <code>microservice2-image:1.0</code>, respectively. Both services are set to restart always, making sure if the container fails it restarts.</p>
<p>Now, build the images using the following command:</p>
<pre><code class="lang-bash">docker compose build
</code></pre>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xhxtoziuxi62finv5hjc.png" alt="Docker Compose build output" /></p>
<hr />
<h2 id="heading-deployment-on-kubernetes">Deployment on Kubernetes 🧑‍🚀</h2>
<blockquote>
<p>Make sure <a target="_blank" href="https://github.com/kubernetes/minikube"><strong>Minikube</strong></a> is installed, or follow <a target="_blank" href="https://minikube.sigs.k8s.io/docs/start/"><strong>this</strong></a> link for installation instructions. 👀</p>
</blockquote>
<p>Create a new local Kubernetes cluster, by running the following command. We will need it when setting up Odigos and Jaeger.</p>
<p><strong>Start Minikube: 🚀</strong></p>
<pre><code class="lang-bash">minikube start
</code></pre>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o6bbtf5jeu64dll2uzgj.png" alt="Starting Minikube" /></p>
<p>Since we are running on a local Kubernetes environment, we need to point our shell to use the minikube's docker-daemon.</p>
<p>To point your shell to minikube's docker-daemon, run:</p>
<pre><code class="lang-bash">minikube -p minikube docker-env | <span class="hljs-built_in">source</span>
</code></pre>
<p>And now, when running any Docker commands such as <code>docker images</code> or <code>docker ps</code>, you will see what is inside Minikube rather than what you have locally on your system.</p>
<p>Now that we have both of our microservices ready and dockerized, it's time to set up Kubernetes for managing these services.</p>
<p>At the root of the project, create a new folder <code>/k8s/manifests</code>. Inside this folder, we will add deployment and service configurations for both of our microservices.</p>
<ul>
<li><p><strong>Deployment Configuration 📜</strong>: For actually deploying the containers on the Kubernetes Cluster.</p>
</li>
<li><p><strong>Service Configuration 📄</strong>: To expose the pods to both within the cluster and outside the cluster.</p>
</li>
</ul>
<p>First, let's create the manifest for the <code>microservice1</code>. Create a new file <code>microservice1-deployment-service.yaml</code> and add the following content:</p>
<pre><code class="lang-yaml"><span class="hljs-string">//</span> <span class="hljs-string">👇</span> <span class="hljs-string">k8s/manifests/microservice1-deployment-service.yaml</span>
<span class="hljs-attr">version:</span> <span class="hljs-string">apps/v1</span>
<span class="hljs-attr">kind:</span> <span class="hljs-string">Deployment</span>
<span class="hljs-attr">metadata:</span>
  <span class="hljs-attr">name:</span> <span class="hljs-string">microservice1</span>
<span class="hljs-attr">spec:</span>
  <span class="hljs-attr">selector:</span>
    <span class="hljs-attr">matchLabels:</span>
      <span class="hljs-attr">app:</span> <span class="hljs-string">microservice1</span>
  <span class="hljs-attr">template:</span>
    <span class="hljs-attr">metadata:</span>
      <span class="hljs-attr">labels:</span>
        <span class="hljs-attr">app:</span> <span class="hljs-string">microservice1</span>
    <span class="hljs-attr">spec:</span>
      <span class="hljs-attr">containers:</span>
        <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">microservice1</span>
          <span class="hljs-attr">image:</span> <span class="hljs-string">microservice1-image:1.0</span>
          <span class="hljs-attr">imagePullPolicy:</span> <span class="hljs-string">Never</span>
          <span class="hljs-attr">resources:</span>
            <span class="hljs-attr">limits:</span>
              <span class="hljs-attr">memory:</span> <span class="hljs-string">"200Mi"</span>
              <span class="hljs-attr">cpu:</span> <span class="hljs-string">"500m"</span>
          <span class="hljs-attr">ports:</span>
            <span class="hljs-bullet">-</span> <span class="hljs-attr">containerPort:</span> <span class="hljs-number">5000</span>
<span class="hljs-meta">---</span>
<span class="hljs-attr">apiVersion:</span> <span class="hljs-string">v1</span>
<span class="hljs-attr">kind:</span> <span class="hljs-string">Service</span>
<span class="hljs-attr">metadata:</span>
  <span class="hljs-attr">name:</span> <span class="hljs-string">microservice1</span>
  <span class="hljs-attr">labels:</span>
    <span class="hljs-attr">app:</span> <span class="hljs-string">microservice1</span>
<span class="hljs-attr">spec:</span>
  <span class="hljs-attr">type:</span> <span class="hljs-string">NodePort</span>
  <span class="hljs-attr">selector:</span>
    <span class="hljs-attr">app:</span> <span class="hljs-string">microservice1</span>
  <span class="hljs-attr">ports:</span>
    <span class="hljs-bullet">-</span> <span class="hljs-attr">port:</span> <span class="hljs-number">8080</span>
      <span class="hljs-attr">targetPort:</span> <span class="hljs-number">5000</span>
      <span class="hljs-attr">nodePort:</span> <span class="hljs-number">30001</span>
</code></pre>
<p>This configuration deploys a microservice named <strong>microservice1</strong> with resource limits of <strong>200MB memory</strong> 🗃️ and <strong>0.5 CPU cores</strong>. It exposes the microservice internally on port 5000 through a Deployment and externally on <strong>NodePort</strong> 30001 through a Service.</p>
<blockquote>
<p>🤔 Remember the <code>docker-compose build</code> command we used when building our Dockerfiles, especially the image names? We are using the same images to create the containers.</p>
</blockquote>
<p>It is accessible on port 8080 within the cluster. We assume <code>microservice1-image:1.0</code> is locally available with <code>imagePullPolicy: Never</code>. If this is not in place, it would attempt to pull the image from the Docker Hub 🐋 and fail.</p>
<p>Now, let's create the manifest for <strong>microservice2</strong>. Create a new file named <code>microservice2-deployment-service.yaml</code> and add the following content:</p>
<pre><code class="lang-yaml"><span class="hljs-string">//</span> <span class="hljs-string">👇</span> <span class="hljs-string">k8s/manifests/microservice2-deployment-service.yaml</span>
<span class="hljs-attr">apiVersion:</span> <span class="hljs-string">apps/v1</span>
<span class="hljs-attr">kind:</span> <span class="hljs-string">Deployment</span>
<span class="hljs-attr">metadata:</span>
  <span class="hljs-attr">name:</span> <span class="hljs-string">microservice2</span>
<span class="hljs-attr">spec:</span>
  <span class="hljs-attr">selector:</span>
    <span class="hljs-attr">matchLabels:</span>
      <span class="hljs-attr">app:</span> <span class="hljs-string">microservice2</span>
  <span class="hljs-attr">template:</span>
    <span class="hljs-attr">metadata:</span>
      <span class="hljs-attr">labels:</span>
        <span class="hljs-attr">app:</span> <span class="hljs-string">microservice2</span>
    <span class="hljs-attr">spec:</span>
      <span class="hljs-attr">containers:</span>
        <span class="hljs-bullet">-</span> <span class="hljs-attr">name:</span> <span class="hljs-string">microservice2</span>
          <span class="hljs-attr">image:</span> <span class="hljs-string">microservice2-image:1.0</span>
          <span class="hljs-attr">imagePullPolicy:</span> <span class="hljs-string">Never</span>
          <span class="hljs-attr">resources:</span>
            <span class="hljs-attr">limits:</span>
              <span class="hljs-attr">memory:</span> <span class="hljs-string">"200Mi"</span>
              <span class="hljs-attr">cpu:</span> <span class="hljs-string">"500m"</span>
          <span class="hljs-attr">ports:</span>
            <span class="hljs-bullet">-</span> <span class="hljs-attr">containerPort:</span> <span class="hljs-number">5001</span>
<span class="hljs-meta">---</span>
<span class="hljs-attr">apiVersion:</span> <span class="hljs-string">v1</span>
<span class="hljs-attr">kind:</span> <span class="hljs-string">Service</span>
<span class="hljs-attr">metadata:</span>
  <span class="hljs-attr">name:</span> <span class="hljs-string">microservice2</span>
  <span class="hljs-attr">labels:</span>
    <span class="hljs-attr">app:</span> <span class="hljs-string">microservice2</span>
<span class="hljs-attr">spec:</span>
  <span class="hljs-attr">type:</span> <span class="hljs-string">NodePort</span>
  <span class="hljs-attr">selector:</span>
    <span class="hljs-attr">app:</span> <span class="hljs-string">microservice2</span>
  <span class="hljs-attr">ports:</span>
    <span class="hljs-bullet">-</span> <span class="hljs-attr">port:</span> <span class="hljs-number">8081</span>
      <span class="hljs-attr">targetPort:</span> <span class="hljs-number">5001</span>
      <span class="hljs-attr">nodePort:</span> <span class="hljs-number">30002</span>
</code></pre>
<p>It is similar to the manifest for <code>microservice1</code>, with just a few changes.</p>
<p>This configuration deploys a microservice named <strong>microservice2</strong> and exposes it internally on port 5001 through a Deployment and externally on <strong>NodePort</strong> 30002 through a Service.</p>
<p>Accessible on port 8081 within the cluster, assuming the <code>microservice2-image:1.0</code> is locally available with <code>imagePullPolicy: Never</code>.</p>
<p>Once, this is all done, make sure to apply these configurations and start the Kubernetes cluster with these services. Change the directory to <code>/manifests</code> and execute the following commands:</p>
<pre><code class="lang-bash">kubectl apply -f microservice1-deployment-service.yaml
kubectl apply -f microservice2-deployment-service.yaml
</code></pre>
<p>Check that both our deployments are <strong>Running</strong> by executing the following command:</p>
<pre><code class="lang-bash">kubectl get pods
</code></pre>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mad4wzglev0ur2f4rpsl.png" alt="Kubernetes Pods" /></p>
<p>Finally, our application is ready and deployed on Kubernetes with the necessary deployment configurations. 🎊</p>
<hr />
<h2 id="heading-installing-odigos">Installing Odigos 🧑‍💻</h2>
<blockquote>
<p>💡 <a target="_blank" href="https://odigos.io/"><strong>Odigos</strong></a> is an open-source observability control plane that enables organizations to create and maintain their observability pipeline. In short, we will use Odigos to auto-instrument our Python application.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kinajglw5njvt49bda98.jpg" alt="Odigos - Monitoring Tool" /></p>
<blockquote>
<p>ℹ️ If you are running on a Mac run the following command to install Odigos locally.</p>
</blockquote>
<pre><code class="lang-bash">brew install keyval-dev/homebrew-odigos-cli/odigos
</code></pre>
<blockquote>
<p>ℹ️ If you are on a Linux machine, consider installing it from GitHub releases by executing the following commands. Make sure to change the file according to your Linux distribution.</p>
<p>ℹ️ If the Odigos binary is not executable, run this command <code>chmod +x odigos</code> to make it executable before running the install command.</p>
</blockquote>
<pre><code class="lang-bash">curl -LJO https://github.com/keyval-dev/odigos/releases/download/v1.0.15/cli_1.0.15_linux_amd64.tar.gz
tar -xvzf cli_1.0.15_linux_amd64.tar.gz
./odigos install
</code></pre>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/41v3a8n4b75v2enwaldr.png" alt="Odigos Installation" /></p>
<blockquote>
<p>If you need more brief instructions on its installation, follow this <a target="_blank" href="https://docs.odigos.io/installation"><strong>link</strong></a>.</p>
</blockquote>
<p>Now, Odigos is ready to run 🚀. We can execute its UI, configure the tracing backend, and send traces accordingly.</p>
<hr />
<h2 id="heading-connecting-odigos-to-a-tracing-backend">Connecting Odigos to a Tracing Backend ✨</h2>
<blockquote>
<p>💡 <a target="_blank" href="https://github.com/jaegertracing/jaeger"><strong>Jaeger</strong></a> is an open source, end-to-end distributed tracing system.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/44r3z7w35hlib6yvuup3.jpg" alt="Jaeger - Distributed Tracing Platform" /></p>
<h3 id="heading-setting-up-jaeger">Setting up Jaeger ✨</h3>
<p>For this tutorial, we will use <strong>Jaeger</strong> 🕵️‍♂️, a popular open-source platform for viewing distributed traces in a microservices application. We will use it to view the traces generated by Odigos.</p>
<blockquote>
<p>For Jaeger installation instructions, follow this <a target="_blank" href="https://www.jaegertracing.io/download/"><strong>link</strong></a>. 👀</p>
</blockquote>
<p>To deploy Jaeger on a Kubernetes cluster, run the following commands:</p>
<pre><code class="lang-bash">kubectl create ns tracing
kubectl apply -f https://raw.githubusercontent.com/keyval-dev/opentelemetry-go-instrumentation/master/docs/getting-started/jaeger.yaml -n tracing
</code></pre>
<p>Here, we are creating a <code>tracing</code> namespace and applying the deployment configuration 📃 for Jaeger in that namespace.</p>
<p>This command sets up the self-hosted Jaeger instance and its service.</p>
<p>Run the below command to get the status of the running pods:</p>
<pre><code class="lang-bash">kubectl get pods -A -w
</code></pre>
<p>Wait for all three pods to be <strong>Running</strong> before proceeding further.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o7et78uptt1lmhekjvgv.png" alt="Kubernetes Pods" /></p>
<p>Now, to view the Jaeger Interface locally, we need to port forward. Forward traffic from port 16686 on the local machine to port 16686 on the selected pod within the Kubernetes cluster.</p>
<pre><code class="lang-bash">kubectl port-forward -n tracing svc/jaeger 16686:16686
</code></pre>
<p>This command creates a tunnel between the local machine and the Jaeger pod, exposing the Jaeger UI so you can interact with it.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/root5frtl55cdnq6ap1h.png" alt="Jaeger Port forwarding" /></p>
<p>Now, on <a target="_blank" href="http://localhost:16686"><code>http://localhost:16686</code></a>, you should be able to see the Jaeger instance running.</p>
<h3 id="heading-configuring-odigos-to-work-with-jaeger">Configuring Odigos to Work with Jaeger 🌟</h3>
<blockquote>
<p>ℹ️ For Linux users, go to the folder where you downloaded the Odigos binaries from GitHub releases and run the following command to launch the Odigos UI.</p>
</blockquote>
<pre><code class="lang-bash">./odigos ui
</code></pre>
<blockquote>
<p>ℹ️ For Mac users, just run:</p>
</blockquote>
<pre><code class="lang-bash">odigos ui
</code></pre>
<p>Visit <a target="_blank" href="http://localhost:3000"><code>http://localhost:3000</code></a> and you will be presented with the Odigos interface where you will see both deployments in the <code>default</code> namespace.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2fk7m0mizp0zl9g1j468.png" alt="Odigos Initial UI" /></p>
<p>Select both of these and click Next. On the next page, choose Jaeger as the backend, and add the following details when prompted:</p>
<ul>
<li><p><strong>Destination Name</strong>: Give any name you want, let's say <strong>python-tracing</strong>.</p>
</li>
<li><p><strong>Endpoint 🎯</strong>: Add <code>jaeger.tracing:4317</code> for the endpoint.</p>
</li>
</ul>
<p>And that's it — Odigos is all set to send traces to our Jaeger backend. It's that simple.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ot8wcuihqrsm9nmz2wcx.png" alt="Odigos tool setup to send traces to Jaeger Tracing" /></p>
<hr />
<h2 id="heading-viewing-distributed-tracing">Viewing Distributed Tracing 🧐</h2>
<p>Odigos has already begun sending traces of our application to Jaeger as soon as we set up Odigos to work with Jaeger as our tracing backend.</p>
<p>Visit <a target="_blank" href="http://localhost:16686"><code>http://localhost:16686</code></a> and select both our microservices in the dropdown.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9idl3pa7lcp5yz4uk36u.png" alt="Jaeger UI dropdown for selecting microservices" /></p>
<p>Make a few requests to our endpoints, and eventually, Jaeger will begin to populate with traces.</p>
<p>Click on any of the requests and explore the traces.</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rzmx6i9p476oj7nm6wj6.png" alt="Jaeger Request Traces" /></p>
<p>This was all done without changing a single line of code. 🔥</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yzjxzhoqamfdvr2lypqj.gif" alt="Wow GIF" /></p>
<hr />
<h2 id="heading-wrap-up">Wrap-Up! ⚡</h2>
<p>So far, you've learned to closely monitor 👀 your Python application with distributed tracing, using <strong>Odigos</strong> as the <strong>middleware</strong> between your application and the tracing backend <strong>Jaeger</strong>.</p>
<p>The source code for this tutorial is available here:</p>
<p><a target="_blank" href="https://github.com/shricodev/blogs/tree/main/odgs-monitor-PY-like-a-pro">https://github.com/shricodev/blogs/tree/main/odgs-monitor-PY-like-a-pro</a></p>
<p>Thank you so much for reading! 🎉 🫡</p>
<blockquote>
<p>Drop down your thoughts in the comment section below. 👇</p>
</blockquote>
<p>Follow me on Socials 🐥: https://linktr.ee/shricodev</p>
]]></content:encoded></item><item><title><![CDATA[Automate your Instagram Posts like a PRO with Cron jobs! 🚀]]></title><description><![CDATA[In this easy-to-follow tutorial, you will learn how to build your own Instagram Automation tool from scratch with cron jobs. 😎 What you will learn: 👀…]]></description><link>https://shricodev.hashnode.dev/automate-your-instagram-posts-like-a-pro-with-cron-jobs</link><guid isPermaLink="true">https://shricodev.hashnode.dev/automate-your-instagram-posts-like-a-pro-with-cron-jobs</guid><category><![CDATA[Python]]></category><category><![CDATA[cronjob]]></category><category><![CDATA[logging]]></category><category><![CDATA[instagram]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Sat, 10 Aug 2024 14:24:27 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642617520/487ee2a7-7618-4242-9439-e3f87004fdc2.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR ✨</h2>
<p>In this easy-to-follow tutorial, you will learn how to build your own Instagram Automation tool from scratch with cron jobs. 😎</p>
<p><strong>What you will learn: 👀</strong></p>
<ul>
<li><p>Learn how to set up <strong>logging</strong> in a Python project.</p>
</li>
<li><p>Learn to add cron jobs in Unix based OS using the <strong>python-crontab</strong> module.</p>
</li>
<li><p>Learn how to post on Instagram with the <strong>instagrapi</strong> module.</p>
</li>
</ul>
<p>So, are you ready to build the coolest Instagram automation tool? 😉</p>
<p><img src="https://i.giphy.com/media/v1.Y2lkPTc5MGI3NjExenA5eDcwZGdzYW1rOHVldnZ1dGplMTdhZXdxMXJkMnpmdW9qenc2ayZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/t3sZxY5zS5B0z5zMIz/giphy-downsized-large.gif" alt="Ready GIF" /></p>
<hr />
<h2 id="heading-setting-up-the-environment">Setting Up the Environment ⚙️</h2>
<p>Before we dive any deeper into building the project, take a look at the project architecture to get a brief idea of the layout.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642614089/07788018-30b9-481a-8f3b-4ba0c7b679a6.png" alt="Project Architecture" /></p>
<blockquote>
<p>💁 We are going to build this project from scratch, making it production-ready with logging support and everything structured in classes and functions.</p>
</blockquote>
<h3 id="heading-initializing-the-project">Initializing the Project 🛠️</h3>
<p>Create a folder to keep all your source code for the project:</p>
<pre><code class="lang-bash">mkdir insta-cron-post-automation
<span class="hljs-built_in">cd</span> insta-cron-post-automation
</code></pre>
<p>Create a few new subfolders where we will store the post data, logs, and shell scripts:</p>
<pre><code class="lang-bash">mkdir -p data logs src/scripts
</code></pre>
<p>Now that the initial folder structure is set up, it's time to create a new virtual environment and install all the modules we will be using in our project.</p>
<p>Run these commands to create and activate a new virtual environment in the root of our project:</p>
<pre><code class="lang-bash">python3 -m venv .venv
<span class="hljs-built_in">source</span> .venv/bin/activate <span class="hljs-comment"># If you are using fish shell, change the activate binary to activate.fish</span>
</code></pre>
<p>Run this command to install all the necessary modules we will be using in our project:</p>
<pre><code class="lang-bash">pip3 install instagrapi python-crontab python-dotenv lorem numpy pillow
</code></pre>
<p>Here is what each module is used for:</p>
<ul>
<li><p><code>instagrapi</code>: Login and post to Instagram.</p>
</li>
<li><p><code>python-crontab</code>: Create and edit the user's crontable.</p>
</li>
<li><p><code>python-dotenv</code>: Read environment variables from the <code>.env</code> file.</p>
</li>
</ul>
<p><strong>Optional Modules</strong></p>
<ul>
<li><p><code>lorem</code>: Generate dummy description for creating sample posts.</p>
</li>
<li><p><code>numpy</code>: Generate random pixel data for creating images for our sample posts.</p>
</li>
<li><p><code>pillow</code>: Create sample images using the pixel data from NumPy.</p>
</li>
</ul>
<h2 id="heading-lets-code-it">Let's Code It 💻</h2>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642615950/94ca7118-8e33-4866-9423-515d29225e2b.gif" alt="Fire GIF" /></p>
<h3 id="heading-setting-up-logging">Setting Up Logging 📋</h3>
<blockquote>
<p>💡 Since our tool operates at specific times provided by the user with a cron job, we can’t rely on print statements to log the output. Everything happens in the background, so we need a central place to view the logs of our program such as a log file.</p>
</blockquote>
<p>For logging support, we will use our old Python friend, the <code>logging</code> module.</p>
<p>Inside the <code>src</code> directory, add a file named <code>logger_config.py</code> with the following code:</p>
<blockquote>
<p>💡 Note that I am using the <code>typing</code> module to set types for variables. After working with TypeScript for so long, I can't resist using type definitions 🫠.</p>
</blockquote>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/logger_config.py</span>
<span class="hljs-keyword">import</span> logging


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_logger</span>(<span class="hljs-params">log_file: str</span>) -&gt; logging.Logger:</span>
    <span class="hljs-string">"""
    Creates and configures a logger to log messages to a specified file.

    This function sets up a logger with an INFO logging level, adds a file handler
    to direct log messages to the specified log file, and applies a specific log
    message format.

    Args:
        log_file (str): The path to the log file where log messages will be saved.

    Returns:
        logging.Logger: Configured logger instance.
    """</span>

    <span class="hljs-comment"># Create a logger instance</span>
    logger = logging.getLogger()

    <span class="hljs-comment"># Set the logging level to INFO</span>
    logger.setLevel(logging.INFO)

    <span class="hljs-comment"># Create a file handler to write log messages to the specified file</span>
    file_handler = logging.FileHandler(log_file)

    <span class="hljs-comment"># Define the format for log messages</span>
    formatter = logging.Formatter(
        <span class="hljs-string">"%(asctime)s - %(name)s - %(levelname)s - %(message)s"</span>
    )
    file_handler.setFormatter(formatter)

    <span class="hljs-comment"># Add the file handler to the logger</span>
    logger.addHandler(file_handler)

    <span class="hljs-keyword">return</span> logger
</code></pre>
<p>So, this <code>get_logger()</code> function takes a path to the log file where it needs to store all the logs instead of logging them to the console. It then creates a logger instance and returns it.</p>
<p>Now, with this function set up, we can call it anywhere in our project, and it will maintain the same logging configuration. 😎</p>
<h3 id="heading-implementing-instagram-login">Implementing Instagram Login 🔑</h3>
<p>Create a new file called <code>setup.py</code> inside the <code>src</code> directory with the following code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/setup.py</span>
<span class="hljs-keyword">import</span> logging
<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> sys
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> NoReturn, Tuple

<span class="hljs-keyword">from</span> dotenv <span class="hljs-keyword">import</span> load_dotenv
<span class="hljs-keyword">from</span> instagrapi <span class="hljs-keyword">import</span> Client


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">log_and_exit</span>(<span class="hljs-params">logger: logging.Logger, message: str</span>) -&gt; NoReturn:</span>
    <span class="hljs-string">"""
    Log an error message and exit the program.

    Args:
    - logger (logging.Logger): The logger to use.
    - message (str): The error message to log.
    """</span>
    logger.error(message)
    sys.exit(<span class="hljs-number">1</span>)


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_credentials</span>(<span class="hljs-params">logger: logging.Logger</span>) -&gt; Tuple[str, str]:</span>
    <span class="hljs-string">"""
    Retrieve the username and password from environment variables.

    This function loads the environment variables from a .env file using dotenv,
    then retrieves the username and password from the environment variables.

    Args:
    - logger (logging.Logger): The logger instance to use for logging.

    Returns:
    - Tuple[str, str]: A tuple containing the username and password retrieved from the environment variables.

    Raises:
    - SystemExit: If the username or password environment variable is missing.
    """</span>

    load_dotenv()

    <span class="hljs-comment"># Get the username and password from the environment variables:</span>
    username: str | <span class="hljs-literal">None</span> = os.getenv(<span class="hljs-string">"INSTA_USERNAME"</span>)
    password: str | <span class="hljs-literal">None</span> = os.getenv(<span class="hljs-string">"INSTA_PASSWORD"</span>)

    <span class="hljs-comment"># Check if username or password is None, and raise an exception if so.</span>
    <span class="hljs-keyword">if</span> username <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span> <span class="hljs-keyword">or</span> password <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span>:
        log_and_exit(
            logger=logger,
            message=<span class="hljs-string">"Username or password environment variable is missing"</span>,
        )

    <span class="hljs-keyword">return</span> username, password


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">setup_instagrapi</span>(<span class="hljs-params">logger: logging.Logger</span>) -&gt; Client:</span>
    <span class="hljs-string">"""
    Set up the instagrapi client with the provided username and password.

    This function uses the get_credentials() function to retrieve the username and password,
    then initializes the instagrapi client with the credentials.

    Args:
    - logger (logging.Logger): The logger instance to use for logging.

    Returns:
    - client (instagrapi.Client): The instagrapi client with the provided credentials.

    Raises:
    - SystemExit: If an error occurs while logging in to Instagram.
    """</span>
    username, password = get_credentials(logger=logger)
    client = Client()

    <span class="hljs-keyword">try</span>:
        login_success = client.login(username=username, password=password)

        <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> login_success:
            log_and_exit(logger=logger, message=<span class="hljs-string">"Instagram Login failed"</span>)

        logger.info(<span class="hljs-string">"Instagram Login successful"</span>)

    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        log_and_exit(
            logger=logger, message=<span class="hljs-string">f"An error occurred while trying to login: <span class="hljs-subst">{e}</span>"</span>
        )

    <span class="hljs-keyword">return</span> client
</code></pre>
<p>The <code>get_credentials()</code> function reads the user environment variables and returns them. As you might have guessed, the function requires you to have the <code>INSTA_USERNAME</code> and <code>INSTA_PASSWORD</code> environment variables set up.</p>
<p>Create a new <code>.env</code> file in the root of the project with both variables defined.</p>
<pre><code class="lang-plaintext">INSTA_USERNAME=&lt;your-insta-username&gt;
INSTA_PASSWORD=&lt;your-insta-password&gt;
</code></pre>
<p>The <code>setup_instagrapi()</code> function creates a new Instagrapi client, logs in to Instagram, and returns the client.</p>
<h3 id="heading-defining-classes">Defining Classes 🧩</h3>
<p>We will set up two different classes: <code>Post</code> and <code>PostList</code>. The <code>PostList</code> class will hold multiple <code>Post</code> objects.</p>
<p>Create a new file named <code>post.py</code> inside the <code>src</code> directory with the following code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/post.py</span>
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> Any, Dict, Optional


<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">Post</span>:</span>
    <span class="hljs-string">"""
    Initializes a new instance of the Post class.

    Args:
    - description (str): The description for the post.
    - image_path (str): The path to the image file.
    - post_date (str): The date and time of the post.
    - extra_data (Optional[Dict[str, Any]]): Additional data for the post. Defaults to None.
    """</span>

    ALLOWED_EXTRA_DATA_FIELDS = {
        <span class="hljs-string">"custom_accessibility_caption"</span>,
        <span class="hljs-string">"like_and_view_counts_disabled"</span>,
        <span class="hljs-string">"disable_comments"</span>,
    }

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">
        self,
        description: str,
        image_path: str,
        post_date: str,
        extra_data: Optional[Dict[str, Any]] = None,
    </span>):</span>
        self.image_path = image_path
        self.description = description
        self.post_date = post_date
        self.extra_data = self.validate_extra_data(extra_data=extra_data)

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">validate_extra_data</span>(<span class="hljs-params">
        self, extra_data: Optional[Dict[str, Any]]
    </span>) -&gt; Optional[Dict[str, Any]]:</span>
        <span class="hljs-string">"""
        Validates and filters the extra_data dictionary to ensure it contains only allowed fields.

        Args:
        - extra_data (Optional[Dict[str, Any]]): The extra data dictionary to validate.

        Returns:
        - Optional[Dict[str, Any]]: The validated extra data dictionary, or None if input is None or invalid.
        """</span>
        <span class="hljs-keyword">if</span> extra_data <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span>:
            <span class="hljs-keyword">return</span> <span class="hljs-literal">None</span>

        validated_data = {
            key: extra_data[key]
            <span class="hljs-keyword">for</span> key <span class="hljs-keyword">in</span> extra_data
            <span class="hljs-keyword">if</span> key <span class="hljs-keyword">in</span> self.ALLOWED_EXTRA_DATA_FIELDS
        }

        <span class="hljs-keyword">return</span> validated_data <span class="hljs-keyword">if</span> validated_data <span class="hljs-keyword">else</span> <span class="hljs-literal">None</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">serialize</span>(<span class="hljs-params">self</span>) -&gt; Dict[str, Any]:</span>
        <span class="hljs-string">"""
        Serialize the object into a dictionary representation.

        Returns:
        - dict: A dictionary containing the serialized data of the object.
                The dictionary has the following keys:
                - "image_path" (str): The path to the image file.
                - "description" (str): The description for the post.
                - "post_date" (str): The date and time of the post.
                If the object has extra data, it is added to the dictionary under the key "extra_data".
        """</span>
        data: Dict[str, Any] = {
            <span class="hljs-string">"image_path"</span>: self.image_path,
            <span class="hljs-string">"description"</span>: self.description,
            <span class="hljs-string">"post_date"</span>: self.post_date,
        }

        <span class="hljs-keyword">if</span> self.extra_data <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span>:
            data[<span class="hljs-string">"extra_data"</span>] = self.extra_data

        <span class="hljs-keyword">return</span> data
</code></pre>
<p>The class takes a few parameters, such as <strong>description</strong> for the post, <strong>image path</strong>, <strong>post date</strong>, and an optional <strong>extra field</strong> property, which can be used to pass additional metadata for the post, like this:</p>
<pre><code class="lang-json">  <span class="hljs-string">"extra_data"</span>: {
    <span class="hljs-attr">"custom_accessibility_caption"</span>: <span class="hljs-string">"An astronaut in the ocean!"</span>,
    <span class="hljs-attr">"like_and_view_counts_disabled"</span>: <span class="hljs-number">0</span>,
    <span class="hljs-attr">"disable_comments"</span>: <span class="hljs-number">1</span>
  },
</code></pre>
<p>Here, the binary values 1 and 0 represent <strong>True</strong> and <strong>False</strong>, respectively.</p>
<p>The <code>validate_extra_data()</code> method checks if the provided <code>extra_data</code> field contains only valid keys and removes any other keys provided by the user.</p>
<p>The <code>serialize()</code> method checks if the <code>extra_data</code> parameter was passed to the constructor. If it was, it adds it to the dictionary and returns the dictionary; otherwise, it returns the dictionary without the <code>extra_data</code> key.</p>
<p>Now that the <code>Post</code> class is ready, let's create another class, <code>PostList</code>, that holds the <code>Post</code> objects.</p>
<p>Create a new file called <code>post_list.py</code> inside the <code>src</code> directory and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/post_list.py</span>
<span class="hljs-keyword">import</span> json
<span class="hljs-keyword">import</span> sys
<span class="hljs-keyword">from</span> datetime <span class="hljs-keyword">import</span> datetime
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> List, NoReturn, Optional

<span class="hljs-keyword">from</span> logger_config <span class="hljs-keyword">import</span> get_logger
<span class="hljs-keyword">from</span> post <span class="hljs-keyword">import</span> Post


<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">PostList</span>:</span>
    <span class="hljs-string">"""
    A class to manage/represent a list of posts.
    """</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, log_path: str</span>):</span>
        self.posts = []
        self.logger = get_logger(log_path)

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">_log_and_exit</span>(<span class="hljs-params">self, message: str</span>) -&gt; NoReturn:</span>
        <span class="hljs-string">"""
        Log an error message and exit the program.

        Args:
        - message (str): The error message to log.
        """</span>
        self.logger.error(message)
        sys.exit(<span class="hljs-number">1</span>)

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">to_json</span>(<span class="hljs-params">self</span>) -&gt; str:</span>
        <span class="hljs-string">"""
        Serialize the list of posts into a JSON string.
        Use this method to write the content in the `self.posts` array to a JSON file.

        Returns:
        - str: JSON string representing the serialized posts.
        """</span>
        serialized_posts = [post.serialize() <span class="hljs-keyword">for</span> post <span class="hljs-keyword">in</span> self.posts]
        <span class="hljs-keyword">return</span> json.dumps({<span class="hljs-string">"posts"</span>: serialized_posts}, default=str)

    <span class="hljs-comment"># Custom function to parse the date without seconds</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">parse_post_date</span>(<span class="hljs-params">self, post_date: str</span>) -&gt; str:</span>
        <span class="hljs-string">"""
        Custom function to parse the date without seconds.

        Args:
        - post_date (str): The date string to parse.

        Returns:
        - str: The parsed date string without seconds.
        """</span>
        date_format = <span class="hljs-string">"%Y-%m-%d %H:%M"</span>

        <span class="hljs-comment"># Parse the date</span>
        parsed_date = datetime.strptime(post_date, date_format)

        <span class="hljs-comment"># Return the date formatted without seconds</span>
        <span class="hljs-keyword">return</span> parsed_date.strftime(<span class="hljs-string">"%Y-%m-%d %H:%M"</span>)

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_posts_from_json_file</span>(<span class="hljs-params">self, posts_file_path: str</span>) -&gt; List[Post]:</span>
        <span class="hljs-string">"""
        Load posts from a JSON file and populate the list.

        Args:
        - posts_file_path (str): The path to the JSON file containing post data.

        Returns:
        - List[Post]: List of Post objects loaded from the JSON file.

        Raises:
        - FileNotFoundError: If the JSON file is not found.
        - PermissionError: If the JSON file cannot be accessed.
        - json.JSONDecodeError: If the JSON file is not valid JSON.
        """</span>
        <span class="hljs-keyword">try</span>:
            <span class="hljs-keyword">with</span> open(posts_file_path, <span class="hljs-string">"r"</span>) <span class="hljs-keyword">as</span> posts_json_file:
                data = json.load(posts_json_file)

                <span class="hljs-keyword">if</span> <span class="hljs-string">"posts"</span> <span class="hljs-keyword">not</span> <span class="hljs-keyword">in</span> data:
                    self._log_and_exit(message=<span class="hljs-string">"No 'posts' key found in the json file"</span>)

                <span class="hljs-keyword">for</span> post <span class="hljs-keyword">in</span> data[<span class="hljs-string">"posts"</span>]:
                    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> all(
                        key <span class="hljs-keyword">in</span> post
                        <span class="hljs-keyword">for</span> key <span class="hljs-keyword">in</span> [<span class="hljs-string">"image_path"</span>, <span class="hljs-string">"description"</span>, <span class="hljs-string">"post_date"</span>]
                    ):
                        self._log_and_exit(
                            message=<span class="hljs-string">"Missing required keys in the post object"</span>
                        )

                    extra_data: Optional[dict] = post.get(<span class="hljs-string">"extra_data"</span>)

                    post_obj = Post(
                        image_path=post[<span class="hljs-string">"image_path"</span>],
                        description=post[<span class="hljs-string">"description"</span>],
                        post_date=self.parse_post_date(post_date=post[<span class="hljs-string">"post_date"</span>]),
                        extra_data=extra_data,
                    )
                    self.posts.append(post_obj)

        <span class="hljs-keyword">except</span> FileNotFoundError:
            self._log_and_exit(message=<span class="hljs-string">f"File not found: <span class="hljs-subst">{posts_file_path}</span>"</span>)

        <span class="hljs-keyword">except</span> PermissionError:
            self._log_and_exit(message=<span class="hljs-string">f"Permission denied: <span class="hljs-subst">{posts_file_path}</span>"</span>)

        <span class="hljs-keyword">except</span> json.JSONDecodeError:
            self._log_and_exit(message=<span class="hljs-string">f"Invalid JSON file: <span class="hljs-subst">{posts_file_path}</span>"</span>)

        <span class="hljs-keyword">except</span> ValueError <span class="hljs-keyword">as</span> ve:
            self._log_and_exit(
                message=<span class="hljs-string">f"Invalid date format provided in the post object: <span class="hljs-subst">{ve}</span>"</span>
            )

        <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
            self._log_and_exit(message=<span class="hljs-string">f"Unexpected error: <span class="hljs-subst">{e}</span>"</span>)

        <span class="hljs-keyword">return</span> self.posts
</code></pre>
<p>The <code>_log_and_exit()</code> method, as the name suggests, is a private method that logs the message to the file and exits the program.</p>
<p>The <code>to_json()</code> method, as the name suggests, returns the list of posts in the form of a JSON string.</p>
<p>The <code>parse_post_date()</code> method takes a <code>post_date</code> variable and returns the date in string format without the seconds portion, as we don't need seconds in cron jobs.</p>
<p>The <code>get_posts_from_json_file()</code> method reads a JSON file, populates the posts array with each post as a <code>Post</code> object, and handles various exceptions that can occur when reading the file contents.</p>
<h3 id="heading-coding-the-media-post-script">Coding the Media Post Script 📝</h3>
<p>Now that we have all the classes set up, it's time to code the main Python script responsible for posting on Instagram.</p>
<p>Create a new file called <code>media_post.py</code> inside the <code>src</code> directory. This file will be quite long, so we'll split the code into each function, and I'll explain it along the way.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/media_post.py</span>
<span class="hljs-keyword">import</span> json
<span class="hljs-keyword">import</span> logging
<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> sys
<span class="hljs-keyword">from</span> datetime <span class="hljs-keyword">import</span> datetime
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> Any, Dict, List, NoReturn, Optional

<span class="hljs-keyword">from</span> instagrapi <span class="hljs-keyword">import</span> Client

<span class="hljs-keyword">from</span> logger_config <span class="hljs-keyword">import</span> get_logger
<span class="hljs-keyword">from</span> setup <span class="hljs-keyword">import</span> setup_instagrapi


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">log_and_exit</span>(<span class="hljs-params">logger: logging.Logger, message: str</span>) -&gt; NoReturn:</span>
    <span class="hljs-string">"""
    Log an error message and exit the program.

    Args:
    - logger (logging.Logger): The logger to use.
    - message (str): The error message to log.
    """</span>
    logger.error(message)
    sys.exit(<span class="hljs-number">1</span>)


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">is_valid_image_extension</span>(<span class="hljs-params">file_name: str</span>) -&gt; bool:</span>
    <span class="hljs-string">"""
    Check if the given file name has a valid image extension.

    Valid extensions are: .jpg, .jpeg, .png.

    Args:
    - file_name (str): The name of the file to check.

    Returns:
    - bool: True if the file has a valid image extension, False otherwise.
    """</span>
    valid_extensions = {<span class="hljs-string">".jpg"</span>, <span class="hljs-string">".jpeg"</span>, <span class="hljs-string">".png"</span>}
    <span class="hljs-keyword">return</span> any(file_name.endswith(ext) <span class="hljs-keyword">for</span> ext <span class="hljs-keyword">in</span> valid_extensions)
</code></pre>
<p>These are fairly straightforward functions. The <code>log_and_exit()</code> function logs the message to the file and exits the program.</p>
<p>The <code>is_valid_image_extension()</code> function checks if the image has a valid extension allowed for posting on Instagram.</p>
<blockquote>
<p>💁 I am not entirely sure if there are other extensions that are allowed, but these seem to be the standard ones. If there are additional extensions, feel free to update them accordingly.</p>
</blockquote>
<p>Once we try to upload the post to Instagram, we need to remove it from the <code>to-post.json</code> file in the <code>data</code> directory, where we add all the posts that we want to schedule. Regardless of whether the upload was successful, we add the post to either the <code>error.json</code> or <code>success.json</code> file inside the <code>data</code> directory.</p>
<p>Create a new function that handles this process.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/media_post.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">handle_post_update</span>(<span class="hljs-params">
    success: bool, json_post_content: Dict[str, Any], logger: logging.Logger
</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Update the post error file based on the success of the upload.

    Args:
    - success (bool): True if the upload was successful, False otherwise.
    - json_post_content (dict): The content of the post.

    Returns:
    - Return the content of the post file if the read is successful; otherwise, return the default value if provided, or None.
    """</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">load_json_file</span>(<span class="hljs-params">file_path: str, default: Optional[Any] = None</span>) -&gt; Any:</span>
        <span class="hljs-string">"""Helper function to load JSON data from a file."""</span>
        <span class="hljs-keyword">if</span> os.path.exists(file_path):
            <span class="hljs-keyword">try</span>:
                <span class="hljs-keyword">with</span> open(file_path, <span class="hljs-string">"r"</span>) <span class="hljs-keyword">as</span> file:
                    <span class="hljs-keyword">return</span> json.load(file)
            <span class="hljs-keyword">except</span> Exception:
                log_and_exit(
                    logger=logger, message=<span class="hljs-string">f"Failed to load post file: <span class="hljs-subst">{file_path}</span>"</span>
                )
        <span class="hljs-keyword">else</span>:
            <span class="hljs-comment"># Create the file with default content if it does not exist</span>
            write_json_file(file_path, default <span class="hljs-keyword">if</span> default <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span> <span class="hljs-keyword">else</span> [])
            <span class="hljs-keyword">return</span> default <span class="hljs-keyword">if</span> default <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span> <span class="hljs-keyword">else</span> []

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">write_json_file</span>(<span class="hljs-params">file_path: str, posts: List[Dict[str, Any]]</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
        <span class="hljs-string">"""Helper function to save JSON data to a file."""</span>
        <span class="hljs-keyword">for</span> post <span class="hljs-keyword">in</span> posts:
            <span class="hljs-keyword">if</span> <span class="hljs-string">"post_date"</span> <span class="hljs-keyword">in</span> post:
                <span class="hljs-keyword">try</span>:
                    post_date = datetime.strptime(
                        post[<span class="hljs-string">"post_date"</span>], <span class="hljs-string">"%Y-%m-%d %H:%M:%S"</span>
                    )
                    post[<span class="hljs-string">"post_date"</span>] = post_date.strftime(<span class="hljs-string">"%Y-%m-%d %H:%M"</span>)
                <span class="hljs-keyword">except</span> ValueError:
                    post_date = datetime.strptime(post[<span class="hljs-string">"post_date"</span>], <span class="hljs-string">"%Y-%m-%d %H:%M"</span>)
                    post[<span class="hljs-string">"post_date"</span>] = post_date.strftime(<span class="hljs-string">"%Y-%m-%d %H:%M"</span>)
                <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
                    log_and_exit(
                        logger=logger, message=<span class="hljs-string">f"Failed to parse post date: <span class="hljs-subst">{e}</span>"</span>
                    )

        <span class="hljs-keyword">try</span>:
            <span class="hljs-keyword">with</span> open(file_path, <span class="hljs-string">"w"</span>) <span class="hljs-keyword">as</span> file:
                json.dump(posts, file, indent=<span class="hljs-number">2</span>)
            logger.info(<span class="hljs-string">f"Post file updated: <span class="hljs-subst">{file_path}</span>"</span>)

        <span class="hljs-keyword">except</span> (IOError, json.JSONDecodeError) <span class="hljs-keyword">as</span> e:
            log_and_exit(logger=logger, message=<span class="hljs-string">f"Failed to write post file: <span class="hljs-subst">{e}</span>"</span>)

    <span class="hljs-comment"># Get the directory of the current script</span>
    current_dir = os.path.dirname(os.path.abspath(__file__))

    <span class="hljs-comment"># Define the directory where the data files are located</span>
    data_dir = os.path.join(current_dir, <span class="hljs-string">".."</span>, <span class="hljs-string">"data"</span>)

    <span class="hljs-comment"># Define paths to the success, error, and to-post files</span>
    success_file = os.path.join(data_dir, <span class="hljs-string">"success.json"</span>)
    error_file = os.path.join(data_dir, <span class="hljs-string">"error.json"</span>)
    to_post_file = os.path.join(data_dir, <span class="hljs-string">"to-post.json"</span>)

    <span class="hljs-comment"># Ensure the success and error files exist</span>
    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(success_file):
        write_json_file(success_file, [])

    <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> os.path.exists(error_file):
        write_json_file(error_file, [])

    <span class="hljs-comment"># Load the current 'to-post' data if it exists, otherwise initialize an empty list</span>
    to_post_data = load_json_file(file_path=to_post_file, default={<span class="hljs-string">"posts"</span>: []})

    <span class="hljs-comment"># Determine which file to write to based on the success of the upload</span>
    target_file = success_file <span class="hljs-keyword">if</span> success <span class="hljs-keyword">else</span> error_file

    <span class="hljs-comment"># Load the current content of the target file if it exists, otherwise initialize an empty list</span>
    target_data = load_json_file(file_path=target_file, default=[])

    <span class="hljs-comment"># Append the current post content to the target data</span>
    target_data.append(json_post_content)

    <span class="hljs-comment"># Write the updated target data back to the target file</span>
    write_json_file(file_path=target_file, posts=target_data)

    user_posts = to_post_data[<span class="hljs-string">"posts"</span>]

    <span class="hljs-comment"># Filter the posted post from the 'to-post' data</span>
    <span class="hljs-keyword">if</span> any(post == json_post_content <span class="hljs-keyword">for</span> post <span class="hljs-keyword">in</span> user_posts):
        user_posts = [item <span class="hljs-keyword">for</span> item <span class="hljs-keyword">in</span> user_posts <span class="hljs-keyword">if</span> item != json_post_content]
        to_post_data[<span class="hljs-string">"posts"</span>] = user_posts
        write_json_file(file_path=to_post_file, posts=to_post_data)
</code></pre>
<p>The <code>handle_post_update()</code> function manages the process of updating files that track the success or failure of post uploads. Depending on whether a post upload was successful or not, the function updates either a success file or an error file with the content of the post.</p>
<p>The function uses nested helper functions, <code>load_json_file()</code> and <code>write_json_file()</code>, to handle the loading and saving of JSON data to and from files. <code>load_json_file()</code> reads data from a file, while <code>write_json_file()</code> saves data back to a file, ensuring that the data format is correct.</p>
<p>Finally, the function updates the relevant files by appending the new post content to either <code>data/success.json</code> or <code>data/error.json</code> and removes the posted content from the <code>data/to-post.json</code> file.</p>
<p>Now, we need a function to parse the file content into JSON. If parsing fails, we need a way to handle the error as well.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/media_post.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">parse_post_file_to_json</span>(<span class="hljs-params">post_path: str, logger: logging.Logger</span>) -&gt; Dict[str, Any]:</span>
    <span class="hljs-string">"""
    Parses the content of a post file into a JSON dictionary.

    Args:
    - post_path (str): The path to the post file.
    - logger (logging.Logger): The logger instance to use for logging errors.

    Returns:
    - Dict[str, Any]: The content of the post file parsed as a JSON dictionary.

    Raises:
    - SystemExit: Exits the program with an error status if the file does not exist,
                  if permission is denied, if JSON decoding fails, or if any other
                  exception occurs during file reading.
    """</span>
    <span class="hljs-keyword">try</span>:
        <span class="hljs-keyword">with</span> open(post_path, <span class="hljs-string">"r"</span>) <span class="hljs-keyword">as</span> post_file:
            content = post_file.read()
        <span class="hljs-keyword">return</span> json.loads(content)

    <span class="hljs-keyword">except</span> FileNotFoundError:
        log_and_exit(logger=logger, message=<span class="hljs-string">f"Post file '<span class="hljs-subst">{post_path}</span>' does not exist"</span>)

    <span class="hljs-keyword">except</span> PermissionError:
        log_and_exit(
            logger=logger,
            message=<span class="hljs-string">f"Permission denied when trying to access post file '<span class="hljs-subst">{post_path}</span>'"</span>,
        )

    <span class="hljs-keyword">except</span> json.JSONDecodeError:
        log_and_exit(
            logger=logger, message=<span class="hljs-string">f"Failed to decode JSON from post file '<span class="hljs-subst">{post_path}</span>'"</span>
        )

    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        log_and_exit(
            logger=logger, message=<span class="hljs-string">f"Failed to read post file '<span class="hljs-subst">{post_path}</span>': <span class="hljs-subst">{e}</span>"</span>
        )


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">handle_post_error</span>(<span class="hljs-params">
    error_message: str, json_post_content: Dict[str, Any], logger: logging.Logger
</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    This function logs an error message, updates the post files to indicate failure,
    and terminates the program with an exit status of 1.

    Args:
    - error_message (str): The error message to be logged.
    - json_post_content (Dict[str, Any]): The content of the post file in JSON format.
    - logger (logging.Logger): The logger instance to use for logging the error.

    Returns:
    - None

    Raises:
    - SystemExit: The program will exit with an exit status of 1.
    """</span>
    handle_post_update(
        success=<span class="hljs-literal">False</span>, json_post_content=json_post_content, logger=logger
    )
    log_and_exit(logger=logger, message=error_message)
</code></pre>
<p>The <code>parse_post_file_to_json()</code> function takes a path to a JSON file and tries to parse its content into JSON. If parsing fails, the <code>handle_invalid_post_file()</code> function is used to handle the failure. It sets the success boolean to <code>false</code>, updates the <code>data/error.json</code> file, and removes the specific post from the <code>data/to-post.json</code> file.</p>
<p>Now that all of this is done, we are finally ready to compute the final upload parameters and upload the post to Instagram.</p>
<p>Add these two functions:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/media_post.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">prepare_upload_params</span>(<span class="hljs-params">
    json_post_content: Dict[str, Any], logger: logging.Logger
</span>) -&gt; Dict[str, Any]:</span>
    <span class="hljs-comment"># Initial needed upload parameters</span>
    upload_params = {
        <span class="hljs-string">"path"</span>: json_post_content.get(<span class="hljs-string">"image_path"</span>),
        <span class="hljs-string">"caption"</span>: json_post_content.get(<span class="hljs-string">"description"</span>),
    }

    <span class="hljs-comment"># If the optional field is provided</span>
    <span class="hljs-keyword">if</span> <span class="hljs-string">"extra_data"</span> <span class="hljs-keyword">in</span> json_post_content:
        extra_data = json_post_content[<span class="hljs-string">"extra_data"</span>]
        <span class="hljs-keyword">try</span>:
            extra_data[<span class="hljs-string">"custom_accessibility_caption"</span>] = str(
                extra_data.get(<span class="hljs-string">"custom_accessibility_caption"</span>, <span class="hljs-string">""</span>)
            )
            extra_data[<span class="hljs-string">"like_and_view_counts_disabled"</span>] = int(
                extra_data.get(<span class="hljs-string">"like_and_view_counts_disabled"</span>, <span class="hljs-number">0</span>)
            )
            extra_data[<span class="hljs-string">"disable_comments"</span>] = int(extra_data.get(<span class="hljs-string">"disable_comments"</span>, <span class="hljs-number">0</span>))

        <span class="hljs-keyword">except</span> (ValueError, TypeError):
            handle_post_error(
                error_message=<span class="hljs-string">f"Failed to parse 'extra_data' field: <span class="hljs-subst">{json_post_content}</span>"</span>,
                json_post_content=json_post_content,
                logger=logger,
            )

        extra_data[<span class="hljs-string">"like_and_view_counts_disabled"</span>] = max(
            <span class="hljs-number">0</span>, min(<span class="hljs-number">1</span>, extra_data[<span class="hljs-string">"like_and_view_counts_disabled"</span>])
        )
        extra_data[<span class="hljs-string">"disable_comments"</span>] = max(<span class="hljs-number">0</span>, min(<span class="hljs-number">1</span>, extra_data[<span class="hljs-string">"disable_comments"</span>]))
        upload_params[<span class="hljs-string">"extra_data"</span>] = extra_data

    <span class="hljs-keyword">return</span> upload_params


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">upload_to_instagram</span>(<span class="hljs-params">
    client: Client,
    upload_params: Dict[str, Any],
    json_post_content: Dict[str, Any],
    logger: logging.Logger,
</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Uploads media to Instagram and handles logging and updating post files based on the result.

    Args:
    - client: The Instagram client used for uploading media.
    - upload_params (Dict[str, Any]): The parameters for the media upload.
    - json_post_content (Dict[str, Any]): The content of the post file in JSON format.
    - logger (logging.Logger): The logger instance to use for logging errors and success messages.

    Returns:
    - None

    Raises:
    - SystemExit: Exits the program with an error status if the upload fails.
    """</span>
    <span class="hljs-keyword">try</span>:
        <span class="hljs-comment"># Upload the media to Instagram</span>
        upload_media = client.photo_upload(**upload_params)

        <span class="hljs-comment"># Get the uploaded post ID</span>
        uploaded_post_id = upload_media.model_dump().get(<span class="hljs-string">"id"</span>, <span class="hljs-literal">None</span>)
        logger.info(
            <span class="hljs-string">f"Successfully uploaded the post on Instagram. ID: <span class="hljs-subst">{uploaded_post_id}</span>"</span>
        )
        handle_post_update(
            success=<span class="hljs-literal">True</span>, json_post_content=json_post_content, logger=logger
        )
    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        handle_post_error(
            error_message=<span class="hljs-string">f"Failed to upload the post: <span class="hljs-subst">{e}</span>"</span>,
            json_post_content=json_post_content,
            logger=logger,
        )
</code></pre>
<p>The <code>prepare_upload_params()</code> function takes the post content and prepares the upload parameters. It includes explicit validation for the <code>extra_data</code> fields to ensure that all keys are of the expected type, and finally returns the entire set of upload parameters.</p>
<p>The <code>upload_to_instagram()</code> function uploads media to Instagram using the provided client and <code>upload_params</code>. If the upload is successful, it logs the post ID and updates the post status using the <code>handle_post_update()</code> function.</p>
<p>If an error occurs during the upload, it logs the error and calls <code>handle_post_error()</code> to handle the failure.</p>
<p>Now, finally, write the main function for the <code>src/media_post.py</code> file:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/src/media_post.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span>() -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Main function to handle the posting process.

    - Sets up logging.
    - Checks if a post file path is provided and valid.
    - Reads and parses the post file.
    - Validates the image file extension.
    - Prepares upload parameters.
    - Logs the upload parameters and response.
    """</span>

    <span class="hljs-comment"># Get the current directory of this script</span>
    current_dir = os.path.dirname(os.path.abspath(__file__))

    <span class="hljs-comment"># Path to the log file, assuming 'logs' is one level up from the current directory</span>
    log_path = os.path.join(current_dir, <span class="hljs-string">".."</span>, <span class="hljs-string">"logs"</span>, <span class="hljs-string">"post-activity.log"</span>)
    logger = get_logger(log_file=log_path)

    <span class="hljs-keyword">if</span> len(sys.argv) &gt; <span class="hljs-number">1</span>:
        post_path = sys.argv[<span class="hljs-number">1</span>]

        <span class="hljs-comment"># Set up the instagrapi client</span>
        client = setup_instagrapi(logger=logger)

        json_post_content: Dict[str, Any] = parse_post_file_to_json(
            post_path=post_path, logger=logger
        )

        <span class="hljs-comment"># If the path does not exist or the path is not a file</span>
        <span class="hljs-keyword">if</span> (<span class="hljs-keyword">not</span> os.path.exists(post_path)) <span class="hljs-keyword">or</span> (<span class="hljs-keyword">not</span> os.path.isfile(post_path)):
            <span class="hljs-keyword">return</span> handle_post_error(
                error_message=<span class="hljs-string">f"'<span class="hljs-subst">{post_path}</span>' does not exist or is not a file"</span>,
                json_post_content=json_post_content,
                logger=logger,
            )

        image_path = json_post_content[<span class="hljs-string">"image_path"</span>]

        <span class="hljs-comment"># Validate image file extension</span>
        <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> is_valid_image_extension(image_path):
            <span class="hljs-keyword">return</span> handle_post_error(
                error_message=<span class="hljs-string">f"'<span class="hljs-subst">{image_path}</span>' is not a valid image"</span>,
                json_post_content=json_post_content,
                logger=logger,
            )

        upload_params: Dict[str, Any] = prepare_upload_params(
            json_post_content=json_post_content, logger=logger
        )

        <span class="hljs-comment"># Log the final upload parameters</span>
        logger.info(<span class="hljs-string">f"Posting to Instagram with the following details: <span class="hljs-subst">{upload_params}</span>"</span>)

        upload_to_instagram(
            client=client,
            upload_params=upload_params,
            json_post_content=json_post_content,
            logger=logger,
        )

    <span class="hljs-keyword">else</span>:
        log_and_exit(logger=logger, message=<span class="hljs-string">"Please provide the path to the post file"</span>)


<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
  main()
</code></pre>
<p>We begin by setting up logging and verifying that the post file path exists. We then initialize the Instagrapi client and read the post file's content, checking for validity in both the file path and image extension.</p>
<p>If any issues are detected, such as an invalid file path or unsupported image type, we log them to the log file.</p>
<p>Once validation is complete, the function prepares the upload parameters and uploads them to Instagram. ✨</p>
<h3 id="heading-building-the-shell-script">Building the Shell Script 🧰</h3>
<p>🤔 <strong>Why is there a need to write a shell script?</strong></p>
<blockquote>
<p>We will use a shell script within the Cron job to execute <code>media_post.py</code>, as we need to source the virtual environment before running the Python script since all the modules are installed there. If we didn't need to source our virtual environment, we could directly run the Python script as a Cron job command without writing this shell script.</p>
</blockquote>
<p>Create a new file called <code>run_media_post.sh</code> inside the <code>src/scripts</code> directory with the following lines of code:</p>
<blockquote>
<p>💁 If you are using the fish shell, you can find the same code with the fish syntax <a target="_blank" href="https://github.com/shricodev/insta-cron-post-automation/blob/main/src/scripts/run_media_post.fish">here</a>. Create a new file called <code>run_media_post.fish</code> inside the <code>src/scripts</code> directory, and add the code from the link.</p>
</blockquote>
<pre><code class="lang-bash"><span class="hljs-meta">#!/usr/bin/env bash</span>
<span class="hljs-comment"># Using this above way of writing shebang can have some security concerns.</span>
<span class="hljs-comment"># See this stackoverflow thread: https://stackoverflow.com/a/21614603</span>
<span class="hljs-comment"># Since, I want this script to be portable for most of the users, instead of hardcoding like '#!/usr/bin/bash', I am using this way.</span>

<span class="hljs-comment"># 👇 insta-cron-post-automation/src/scripts/run_media_post.sh</span>

<span class="hljs-comment"># Constants for error messages</span>
ERROR_USAGE=<span class="hljs-string">"ERROR: Usage: bash {media_post_path} {post_file_path}"</span>
ERROR_FILE_NOT_FOUND=<span class="hljs-string">"ERROR: One or both of the files do not exist or are not valid files."</span>
ERROR_PYTHON_NOT_FOUND=<span class="hljs-string">"ERROR: No suitable Python executable found."</span>
ERROR_BASH_NOT_INSTALLED=<span class="hljs-string">"ERROR: Bash shell is not installed. Please install Bash."</span>
ERROR_ACTIVATE_NOT_FOUND=<span class="hljs-string">"ERROR: activate file not found in '<span class="hljs-variable">$VENV_DIR</span>/bin'"</span>
ERROR_UNSUPPORTED_SHELL=<span class="hljs-string">"ERROR: Unsupported shell: '<span class="hljs-variable">$SHELL</span>'"</span>

<span class="hljs-comment"># Determine the script directory and virtual environment directory</span>
SCRIPT_DIR=<span class="hljs-string">"<span class="hljs-subst">$(dirname <span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$0</span>"</span>)</span>"</span>)</span>"</span>
VENV_DIR=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$SCRIPT_DIR</span>/../../.venv"</span>)</span>"</span>
LOG_FILE=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$SCRIPT_DIR</span>/../../logs/shell-error.log"</span>)</span>"</span>

<span class="hljs-function"><span class="hljs-title">log_and_exit</span></span>() {
  <span class="hljs-built_in">local</span> message=<span class="hljs-string">"<span class="hljs-variable">$1</span>"</span>

  <span class="hljs-built_in">echo</span> <span class="hljs-string">"[<span class="hljs-subst">$(date +<span class="hljs-string">"%Y-%m-%d %H:%M:%S"</span>)</span>] <span class="hljs-variable">$message</span>"</span> | tee -a <span class="hljs-variable">$LOG_FILE</span>
  <span class="hljs-built_in">exit</span> 1
}

<span class="hljs-comment"># Check if both arguments are provided</span>
<span class="hljs-keyword">if</span> [ <span class="hljs-variable">$#</span> -ne 2 ]; <span class="hljs-keyword">then</span>
  log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_USAGE</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Function to check if a file exists and has the correct extension</span>
<span class="hljs-function"><span class="hljs-title">check_file</span></span>() {
    <span class="hljs-built_in">local</span> file_path=<span class="hljs-string">"<span class="hljs-variable">$1</span>"</span>
    <span class="hljs-built_in">local</span> expected_extension=<span class="hljs-string">"<span class="hljs-variable">$2</span>"</span>

    <span class="hljs-keyword">if</span> [ ! -f <span class="hljs-string">"<span class="hljs-variable">$file_path</span>"</span> ]; <span class="hljs-keyword">then</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_FILE_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>

    <span class="hljs-keyword">if</span> ! [[ <span class="hljs-string">"<span class="hljs-variable">$file_path</span>"</span> == *<span class="hljs-string">".<span class="hljs-variable">$expected_extension</span>"</span> ]]; <span class="hljs-keyword">then</span>
        log_and_exit <span class="hljs-string">"The file '<span class="hljs-variable">$file_path</span>' must be a '.<span class="hljs-variable">$expected_extension</span>' file."</span>
    <span class="hljs-keyword">fi</span>
}

<span class="hljs-comment"># Validate the provided files</span>
check_file <span class="hljs-string">"<span class="hljs-variable">$1</span>"</span> <span class="hljs-string">"py"</span>
check_file <span class="hljs-string">"<span class="hljs-variable">$2</span>"</span> <span class="hljs-string">"json"</span>

<span class="hljs-comment"># Extract and validate arguments</span>
MEDIA_POST_PATH=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$1</span>"</span>)</span>"</span>
POST_FILE_PATH=<span class="hljs-string">"<span class="hljs-subst">$(realpath <span class="hljs-string">"<span class="hljs-variable">$2</span>"</span>)</span>"</span>

<span class="hljs-comment"># Find the appropriate Python executable</span>
PYTHON_EXEC=<span class="hljs-string">"<span class="hljs-subst">$(command -v python3 || command -v python)</span>"</span>

<span class="hljs-comment"># Ensure that the Python executable is available before creating the virtual environment</span>
<span class="hljs-keyword">if</span> [ ! -d <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>"</span> ]; <span class="hljs-keyword">then</span>
    <span class="hljs-keyword">if</span> [ -z <span class="hljs-string">"<span class="hljs-variable">$PYTHON_EXEC</span>"</span> ]; <span class="hljs-keyword">then</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_PYTHON_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>
    <span class="hljs-string">"<span class="hljs-variable">$PYTHON_EXEC</span>"</span> -m venv <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-keyword">if</span> ! <span class="hljs-built_in">command</span> -v bash &amp;&gt; /dev/null; <span class="hljs-keyword">then</span>
    log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_BASH_NOT_INSTALLED</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Activate the virtual environment based on the shell type</span>
<span class="hljs-keyword">if</span> [[ <span class="hljs-string">"<span class="hljs-variable">$SHELL</span>"</span> == *<span class="hljs-string">"/bash"</span> ]]; <span class="hljs-keyword">then</span>
    <span class="hljs-comment"># Check if the activate file exists before sourcing it</span>
    <span class="hljs-keyword">if</span> [ -f <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>/bin/activate"</span> ]; <span class="hljs-keyword">then</span>
        <span class="hljs-built_in">source</span> <span class="hljs-string">"<span class="hljs-variable">$VENV_DIR</span>/bin/activate"</span>
    <span class="hljs-keyword">else</span>
        log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_ACTIVATE_NOT_FOUND</span>"</span>
    <span class="hljs-keyword">fi</span>
<span class="hljs-keyword">else</span>
    log_and_exit <span class="hljs-string">"<span class="hljs-variable">$ERROR_UNSUPPORTED_SHELL</span>"</span>
<span class="hljs-keyword">fi</span>

<span class="hljs-comment"># Set the python executable to the one from the virtual environment</span>
PYTHON_EXEC=<span class="hljs-string">"<span class="hljs-subst">$(command -v python)</span>"</span>

<span class="hljs-string">"<span class="hljs-variable">$PYTHON_EXEC</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$MEDIA_POST_PATH</span>"</span> <span class="hljs-string">"<span class="hljs-variable">$POST_FILE_PATH</span>"</span>

<span class="hljs-comment"># Remove the cron job after running the script</span>
crontab -l | grep -v <span class="hljs-string">"<span class="hljs-variable">$POST_FILE_PATH</span>"</span> | crontab -
</code></pre>
<p>This script is designed to automate the execution of the Python script <code>media_post.py</code>, which is responsible for uploading content to Instagram with specified arguments, while ensuring that the environment is correctly set up beforehand.</p>
<p>It first checks if the correct number of arguments (two file paths) are provided, then validates that these files exist and have the correct extensions (<strong>.py</strong> for the Python script and <strong>.json</strong> for the post data file).</p>
<p>The script also checks if <strong>Python</strong> and <strong>Bash</strong> are installed on the system and sets up a virtual environment. It supports only the Bash shell and will activate the virtual environment before running the Python script.</p>
<p>After execution, the script removes the cron job that triggered its execution by invert-matching with the <code>grep</code> command.</p>
<h3 id="heading-writing-the-mainpy-file">Writing the <code>main.py</code> File 🧑‍💻</h3>
<p>This is the only Python script that we need to run manually after populating the <code>data/to-post.json</code> file.</p>
<p>We'll write this file in chunks and explain it along the way. Create a new file called <code>main.py</code> inside the root of the project and add the following lines of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/main.py</span>
<span class="hljs-keyword">import</span> json
<span class="hljs-keyword">import</span> logging
<span class="hljs-keyword">import</span> os
<span class="hljs-keyword">import</span> secrets
<span class="hljs-keyword">import</span> string
<span class="hljs-keyword">import</span> sys
<span class="hljs-keyword">from</span> datetime <span class="hljs-keyword">import</span> datetime
<span class="hljs-keyword">from</span> os <span class="hljs-keyword">import</span> environ
<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> Dict, NoReturn

<span class="hljs-keyword">from</span> dateutil <span class="hljs-keyword">import</span> tz

<span class="hljs-comment"># Add the src directory to the module search path</span>
sys.path.insert(<span class="hljs-number">0</span>, os.path.join(os.path.dirname(os.path.abspath(__file__)), <span class="hljs-string">"src"</span>))

<span class="hljs-keyword">from</span> crontab <span class="hljs-keyword">import</span> CronTab

<span class="hljs-keyword">from</span> src <span class="hljs-keyword">import</span> logger_config, post_list


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">log_and_exit</span>(<span class="hljs-params">logger: logging.Logger, message: str</span>) -&gt; NoReturn:</span>
    <span class="hljs-string">"""
    Log an error message and exit the program.

    Args:
    - logger (logging.Logger): The logger to use.
    - message (str): The error message to log.
    """</span>
    logger.error(message)
    sys.exit(<span class="hljs-number">1</span>)


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_shell_script_to_run</span>(<span class="hljs-params">
    user_shell: str, current_dir: str, logger: logging.Logger
</span>) -&gt; str:</span>
    <span class="hljs-string">"""
    Determine the script to run based on the user's shell.

    Args:
    - user_shell (str): The user's shell.
    - current_dir (str): The current directory of the script.
    - logger (logging.Logger): The logger to use.

    Returns:
    - str: The path to the appropriate shell script for the user's shell.

    Raises:
    - SystemExit: If the user's shell is unsupported.
    """</span>

    shell_script_map: Dict[str, str] = {
        <span class="hljs-string">"bash"</span>: os.path.join(current_dir, <span class="hljs-string">"src"</span>, <span class="hljs-string">"scripts"</span>, <span class="hljs-string">"run_media_post.sh"</span>),
        <span class="hljs-string">"fish"</span>: os.path.join(current_dir, <span class="hljs-string">"src"</span>, <span class="hljs-string">"scripts"</span>, <span class="hljs-string">"run_media_post.fish"</span>),
    }

    run_media_post_path = shell_script_map.get(user_shell, <span class="hljs-literal">None</span>)
    <span class="hljs-keyword">if</span> run_media_post_path <span class="hljs-keyword">is</span> <span class="hljs-literal">None</span>:
        log_and_exit(logger=logger, message=<span class="hljs-string">f"Unsupported shell: <span class="hljs-subst">{user_shell}</span>"</span>)

    <span class="hljs-keyword">return</span> run_media_post_path
</code></pre>
<blockquote>
<p>👀 Notice that we are inserting the path to our <code>src</code> directory using the <code>sys.path.insert()</code> method to ensure that Python can locate and import modules from the <code>src</code> directory.</p>
</blockquote>
<p>The <code>log_and_exit()</code> function is the same as before—if something goes wrong, it logs the error and exits the program. The <code>get_shell_script_to_run()</code> function returns the path to the shell script that should be run in the cron job based on whether the user's shell is either <strong>Bash</strong> or <strong>Fish</strong>. If the user's shell is not one of these, the program will exit.</p>
<p>Now, let's add a helper function to validate the post date and add a cron job with the provided arguments.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/main.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">validate_post_date</span>(<span class="hljs-params">post_date: str, logger: logging.Logger</span>) -&gt; datetime:</span>
    <span class="hljs-string">"""
    Validate the post date to ensure it is in the future.

    Args:
    - post_date (string): The date and time of the post.
    - logger (logging.Logger): The logger to use.

    Returns:
    - datetime: The validated and parsed datetime object.

    Raises:
    - SystemExit: If the post date is not valid or not in the future.
    """</span>

    <span class="hljs-comment"># Define the expected format for parsing</span>
    date_format = <span class="hljs-string">"%Y-%m-%d %H:%M"</span>

    <span class="hljs-keyword">try</span>:
        <span class="hljs-comment"># Attempt to parse the post_date string into a datetime object</span>
        parsed_date = datetime.strptime(post_date, date_format)
    <span class="hljs-keyword">except</span> ValueError:
        log_and_exit(
            logger=logger,
            message=<span class="hljs-string">f"The post_date is not in the correct format: <span class="hljs-subst">{post_date}</span>"</span>,
        )

    <span class="hljs-comment"># Check if the parsed date is in the future</span>
    <span class="hljs-keyword">if</span> parsed_date.astimezone(tz.UTC) &lt;= datetime.now(tz=tz.UTC):
        log_and_exit(
            logger=logger, message=<span class="hljs-string">f"The post_date `<span class="hljs-subst">{post_date}</span>` is in the past."</span>
        )

    <span class="hljs-keyword">return</span> parsed_date


<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">create_cron_job</span>(<span class="hljs-params">
    cron: CronTab,
    user_shell: str,
    run_media_post_path: str,
    media_post_path: str,
    scheduled_post_file_path: str,
    post_date: datetime,
    logger: logging.Logger,
</span>) -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Create a cron job for a scheduled post.

    Args:
    - cron (CronTab): The crontab object for the current user.
    - user_shell (str): The user's shell.
    - run_media_post_path (str): The path to the shell script to run.
    - media_post_path (str): The path to the media post script.
    - scheduled_post_file_path (str): The path to the scheduled post file.
    - post_date (datetime): The date and time to run the job.
    - logger (logging.Logger): The logger to use.

    Raises:
    - SystemExit: If the cron job creation fails.
    """</span>
    <span class="hljs-keyword">try</span>:
        <span class="hljs-comment"># Conditionally add semicolon</span>
        command = (
            <span class="hljs-string">f"SHELL=$(command -v <span class="hljs-subst">{user_shell}</span>)"</span>
            + (<span class="hljs-string">";"</span> <span class="hljs-keyword">if</span> user_shell == <span class="hljs-string">"bash"</span> <span class="hljs-keyword">else</span> <span class="hljs-string">""</span>)
            + <span class="hljs-string">f" <span class="hljs-subst">{user_shell}</span> <span class="hljs-subst">{run_media_post_path}</span> <span class="hljs-subst">{media_post_path}</span> <span class="hljs-subst">{scheduled_post_file_path}</span>"</span>
        )
        job = cron.new(command=command)
        job.setall(post_date.strftime(<span class="hljs-string">"%M %H %d %m *"</span>))
    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        log_and_exit(logger=logger, message=<span class="hljs-string">f"Failed to create cron job: <span class="hljs-subst">{e}</span>"</span>)
</code></pre>
<p>The <code>validate_post_date()</code> function checks whether the datetime string is in the expected format (without seconds) and ensures that the specified post date for Instagram isn’t in the past.</p>
<p>The <code>create_cron_job()</code> function takes a configured <strong>Crontab</strong> object, the path to the shell script, the path to <code>media_post.py</code>, and the path to the file containing the scheduled post content. It then creates a cron job with the <strong>SHELL variable</strong> set to the user's shell, because the cron environment may use a shell different from the current user's shell, and schedules the job to execute at the specified time.</p>
<p>If any exception occurs during the scheduling of the cron job, the function logs the error and exits the program.</p>
<p>Now, it's time to code the main function responsible for setting everything up:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 insta-cron-post-automation/main.py</span>

<span class="hljs-comment"># Rest of the code...</span>

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span>() -&gt; <span class="hljs-keyword">None</span>:</span>
    <span class="hljs-string">"""
    Main function to schedule Instagram posts using cron jobs.

    This function performs the following tasks:
    1. Sets up logging to a file.
    2. Loads a list of posts from a JSON file.
    3. Creates a temporary JSON file for each post to be scheduled.
    4. Schedules a cron job to execute a script for each post at the specified date and time.
    5. Writes the cron jobs to the user's crontab.

    The cron job will execute the script `media_post.py` with the path to the temporary JSON file as an argument.
    """</span>

    <span class="hljs-comment"># Determine the current directory of the script</span>
    current_dir = os.path.dirname(os.path.abspath(__file__))

    <span class="hljs-comment"># Define paths for log file and posts JSON file</span>
    log_path = os.path.join(current_dir, <span class="hljs-string">"logs"</span>, <span class="hljs-string">"post-activity.log"</span>)
    to_post_path = os.path.join(current_dir, <span class="hljs-string">"data"</span>, <span class="hljs-string">"to-post.json"</span>)
    media_post_path = os.path.join(current_dir, <span class="hljs-string">"src"</span>, <span class="hljs-string">"media_post.py"</span>)

    <span class="hljs-comment"># Initialize logger</span>
    logger = logger_config.get_logger(log_file=log_path)

    post_data_dir = os.path.join(current_dir, <span class="hljs-string">"data"</span>, <span class="hljs-string">"scheduled_posts"</span>)
    os.makedirs(post_data_dir, exist_ok=<span class="hljs-literal">True</span>)

    <span class="hljs-comment"># Initialize PostList object and load posts from JSON file</span>
    posts_list = post_list.PostList(log_path)

    posts_list.get_posts_from_json_file(posts_file_path=to_post_path)
    logger.info(<span class="hljs-string">f"Number of posts loaded: <span class="hljs-subst">{len(posts_list.posts)}</span>"</span>)

    user_shell = os.path.basename(environ.get(<span class="hljs-string">"SHELL"</span>, <span class="hljs-string">"/bin/bash"</span>))
    run_media_post_path = get_shell_script_to_run(
        user_shell=user_shell, current_dir=current_dir, logger=logger
    )

    <span class="hljs-comment"># Access the current user's CronTab object.</span>
    cron = CronTab(user=<span class="hljs-literal">True</span>)

    <span class="hljs-keyword">for</span> post <span class="hljs-keyword">in</span> posts_list.posts:
        <span class="hljs-comment"># Create a unique identifier for each post file</span>
        unique_id = <span class="hljs-string">""</span>.join(
            secrets.choice(string.ascii_lowercase + string.digits) <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(<span class="hljs-number">6</span>)
        )

        post.post_date = validate_post_date(post_date=post.post_date, logger=logger)

        <span class="hljs-comment"># Create a unique suffix for the temporary file based on the post date</span>
        post_date_suffix = post.post_date.strftime(<span class="hljs-string">"%Y-%m-%d-%H-%M"</span>)

        scheduled_post_file_path = os.path.join(
            post_data_dir, <span class="hljs-string">f"insta_post_<span class="hljs-subst">{unique_id}</span>_<span class="hljs-subst">{post_date_suffix}</span>.json"</span>
        )

        <span class="hljs-comment"># Write the post data to the temporary file</span>
        <span class="hljs-keyword">try</span>:
            <span class="hljs-keyword">with</span> open(scheduled_post_file_path, <span class="hljs-string">"w"</span>) <span class="hljs-keyword">as</span> f:
                json.dump(post.serialize(), f, default=str)
        <span class="hljs-keyword">except</span> (IOError, json.JSONDecodeError) <span class="hljs-keyword">as</span> e:
            log_and_exit(logger=logger, message=<span class="hljs-string">f"Failed to write post file: <span class="hljs-subst">{e}</span>"</span>)

        <span class="hljs-comment"># Create a new cron job to run the Instagram post script with the temp file as an argument</span>
        create_cron_job(
            cron=cron,
            user_shell=user_shell,
            run_media_post_path=run_media_post_path,
            media_post_path=media_post_path,
            scheduled_post_file_path=scheduled_post_file_path,
            post_date=post.post_date,
            logger=logger,
        )

    <span class="hljs-comment"># Write the cron jobs to the user's crontab</span>
    <span class="hljs-keyword">try</span>:
        cron.write()
        logger.info(<span class="hljs-string">f"Cronjob added to the CronTab for the current user: <span class="hljs-subst">{cron.user}</span>"</span>)
    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        log_and_exit(logger=logger, message=<span class="hljs-string">f"Failed to write to CronTab: <span class="hljs-subst">{e}</span>"</span>)


<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    main()
</code></pre>
<p>The <code>main()</code> function sets up a scheduling system for Instagram posts using cron jobs. It begins by configuring logging and loading a list of posts from a JSON file (<code>data/to-post.json</code>). For each post, it creates a JSON file inside the <code>data/scheduled-posts</code> directory with the post content and schedules a cron job to run a script that handles posting at the specified date and time.</p>
<p>It also determines the user's shell and sets up the appropriate script to execute. After creating unique temporary files and scheduling the jobs, it writes all the cron jobs to the user's crontab. If any errors occur during these processes, they are logged, and the program exits.</p>
<hr />
<h2 id="heading-testing-the-program">Testing the Program 🧪</h2>
<p>If you’re curious about how this program works, I’ve prepared a sample script called <code>populate_sample_posts.py</code> that will populate the <code>data/to-post.json</code> file with a sample post, including a description, post date, and an image. You can find it <a target="_blank" href="https://github.com/shricodev/insta-cron-post-automation/blob/main/src/populate_sample_posts.py">here</a>.</p>
<p>After you populate the <code>data/to-post.json</code> file and are inside a virtual environment, run this command:</p>
<pre><code class="lang-bash">python3 main.py
</code></pre>
<p>It’s recommended to test this with a new Instagram account first before using it with your main account. Once you are satisfied, it's time to schedule your own Instagram posts! 😉</p>
<blockquote>
<p><strong>DISCLAIMER ⚠️</strong> This script uses Cron jobs, so it will only be able to schedule your posts if the system is running. Therefore, it's best to run it on a cloud-based VM that is online nearly 24/7.</p>
</blockquote>
<hr />
<h2 id="heading-wrap-up">Wrap-Up! ⚡</h2>
<p>Whoof, 😮‍💨 what a journey it has been! If you've made it this far, give yourself a well-deserved pat on the back. By now, you’ve successfully built a Python application to automate Instagram posting using Cron jobs. 🤯</p>
<p>This has to be one of the coolest and most unique scripts you’ve built with Python.</p>
<p>And I’m pretty sure this is not something you’ll find easily on the Internet. 🥱</p>
<p>The entire documented source code for this article is available here:</p>
<p>https://github.com/shricodev/insta-cron-post-automation</p>
<p>Thank you so much for reading! 🎉 🫡</p>
<blockquote>
<p>Drop down your thoughts in the comment section below. 👇</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[✨ Do this first if you are using an Auth Provider 🧙‍♂️ 🫵]]></title><description><![CDATA[In this article, I will show you how you can safely backup your users data in your own database when using an Authentication Provider. Did you ever…]]></description><link>https://shricodev.hashnode.dev/do-this-first-if-you-are-using-an-auth-provider</link><guid isPermaLink="true">https://shricodev.hashnode.dev/do-this-first-if-you-are-using-an-auth-provider</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Sun, 30 Jun 2024 13:16:24 GMT</pubDate><enclosure url="https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fccsc1b5miw46t6vwftew.gif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In this article, I will show you how you can safely backup your users data in your own database when using an Authentication Provider.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642585152/6aca00df-e751-42a3-8de0-a69f675e9ce2.gif" alt="What GIF" /></p>
<h2 id="heading-why-keep-a-backup-in-the-first-place">Why keep a backup in the first place? 🤔</h2>
<p>Did you ever consciously think that when using an auth provider, you are essentially storing your users' information with them and you do not have access to the user data outside of them (not even in your own database)? 😳</p>
<p>The main purpose of Auth Providers is to abstract away the user authentication logic, but in doing so, you are also completely giving your users' data to them without retaining any control yourself.</p>
<p>What if a new intern joins your auth provider company and mistakenly deletes the production database? This is extremely rare, but the chances are not zero. Not only could they shut down their company, but you would also lose all your users' data. They might have some backup set up to prevent such scenarios, but you never know how things are implemented under the hood in another company.</p>
<p>Many of you don’t even think of this and get started with using one of the providers just because they are <strong>easier</strong> 🤷‍♂️ to get up and running.</p>
<blockquote>
<p>If you are using one then I am pretty sure that you don’t even have a User table in your database. Did I guess right? 🤨</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642587384/976fc5e8-2bc3-43b3-b5d2-598708b73b03.gif" alt="Shocked" /></p>
<p>If this realization hits 🫠, then continue with the article as I show you how to safely keep a backup of the user’s data.</p>
<hr />
<h2 id="heading-setting-up-the-project-with-kinde">Setting up the Project with Kinde 🚀</h2>
<blockquote>
<p>ℹ️ If you already have a project that uses an Auth Provider, feel free to skip this section.</p>
</blockquote>
<p>I will show you an example in a sample <strong>Next.js</strong> application with one of the popular auth provider called <a target="_blank" href="https://kinde.com">Kinde</a>.</p>
<p>The steps are going to be fairly the same when using any other providers as well.</p>
<p>Run the following command to bootstrap a new Next.js application with <strong>Tailwind</strong>, <strong>Eslint</strong> and <strong>Typescript</strong> support:</p>
<pre><code class="lang-bash">bunx create-next-app@latest --tailwind --eslint --typescript
</code></pre>
<blockquote>
<p>The above command uses <a target="_blank" href="https://bun.sh/">bun</a> as the package manager. If you don’t have it installed, you can go ahead with npm, pnpm or yarn.</p>
</blockquote>
<h3 id="heading-setting-up-kinde-authentication">Setting up Kinde Authentication</h3>
<p>Make sure to have the necessary kinde package installed with the below command:</p>
<pre><code class="lang-bash">bun i @kinde-oss/kinde-auth-nextjs
</code></pre>
<p>Create a new project in Kinde and copy all the environment variables into <code>.env</code> file in your project.</p>
<pre><code class="lang-plaintext">KINDE_CLIENT_ID=&lt;your_kinde_client_id&gt;
KINDE_CLIENT_SECRET=&lt;your_kinde_client_secret&gt;
KINDE_ISSUER_URL=https://&lt;your_kinde_subdomain&gt;.kinde.com
KINDE_SITE_URL=http://localhost:3000
KINDE_POST_LOGOUT_REDIRECT_URL=http://localhost:3000
KINDE_POST_LOGIN_REDIRECT_URL=http://localhost:3000/dashboard
</code></pre>
<p>Note the <code>KINDE_POST_LOGIN_REDIRECT_URL</code> variable. This variable ensures that once the user is authenticated in Kinde, they are redirected to the <code>/dashboard</code> endpoint.</p>
<p>Make sure to change it according to your needs. Our code assumes that the user is redirected to <code>/dashboard</code> once they successfully log in.</p>
<p>Now, we need to set up the Kinde Auth Router Handlers. In the <code>app/app/api/auth/[kindeAuth]/route.ts</code>, add the following code:</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> {handleAuth} <span class="hljs-keyword">from</span> <span class="hljs-string">"@kinde-oss/kinde-auth-nextjs/server"</span>;
<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> GET = handleAuth();
</code></pre>
<p>This will setup the necessary route handler to add Kinde Authentication to our application.</p>
<hr />
<h2 id="heading-setting-up-the-database-model">Setting up the Database Model 🛠️</h2>
<blockquote>
<p>ℹ️ I will be using MongoDB as the database and Prisma as the ORM. If you prefer any other Prisma alternatives like Drizzle or Mongoose, feel free to proceed with them.</p>
</blockquote>
<p>Run the following command to install Prisma as a development dependency:</p>
<pre><code class="lang-bash">bun i prisma @prisma/client --save-dev
</code></pre>
<p>Now, initialize Prisma with the following command:</p>
<pre><code class="lang-bash">bunx prisma init
</code></pre>
<p>After you run this command, a new <code>schema.prisma</code> file should be created in the <code>prisma</code> folder at the root of your project.</p>
<p>Modify the <code>schema.prisma</code> file to include a new User model. The fields in the model can vary based on the information your auth provider provides upon successful user creation.</p>
<pre><code class="lang-plaintext">generator client {
  provider = "prisma-client-js"
}

datasource db {
  provider = "mongodb"
  url      = env("DATABASE_URL")
}

model User {
  id       String  @id @map("_id") @db.String
  email    String  @unique
  username String  @unique
  name     String?

  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
}

// If you are adding this change on top of your exising Prisma Schema,
// you will have the rest of your models here...
</code></pre>
<p>Now that we have our model ready, we need to push it to our database. For this, we need the connection URL.</p>
<p>If you already have a connection URL, that's great. If not, and you're following along, create a new cluster in <strong>MongoDB Atlas</strong> and obtain the database connection string. Then, add a new variable named <code>DATABASE_URL</code> with the connection string value to the <code>.env</code> file.</p>
<pre><code class="lang-plaintext">DATABASE_URL=&lt;db-connection-string&gt;

// Rest of the environment variables...
</code></pre>
<p>Now, we need to setup the <code>PrismaClient</code> which we can use to query our database. Create a new file <code>index.ts</code> in the <code>/src/db</code> directory with the following lines of code:</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> { PrismaClient } <span class="hljs-keyword">from</span> <span class="hljs-string">"@prisma/client"</span>;

<span class="hljs-keyword">declare</span> <span class="hljs-built_in">global</span> {
  <span class="hljs-comment">// eslint-disable-next-line no-var</span>
  <span class="hljs-keyword">var</span> cachedPrisma: PrismaClient;
}

<span class="hljs-keyword">let</span> prisma: PrismaClient;

<span class="hljs-keyword">if</span> (process.env.NODE_ENV === <span class="hljs-string">"production"</span>) prisma = <span class="hljs-keyword">new</span> PrismaClient();
<span class="hljs-keyword">else</span> {
  <span class="hljs-keyword">if</span> (!<span class="hljs-built_in">global</span>.cachedPrisma) <span class="hljs-built_in">global</span>.cachedPrisma = <span class="hljs-keyword">new</span> PrismaClient();
  prisma = <span class="hljs-built_in">global</span>.cachedPrisma;
}

<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> db = prisma;
</code></pre>
<p>In development environment, the code initializes <code>PrismaClient</code> once and caches it globally to optimize resource usage. In production, it creates a new <code>PrismaClient</code> instance per request.</p>
<p>Run the following command to push your changes in your schema to the database.</p>
<pre><code class="lang-bash">bunx prisma db push
</code></pre>
<p>Now, to have the updated types work in the IDE, run the following command to generate new types based on our updated schema.</p>
<pre><code class="lang-bash">bunx prisma generate
</code></pre>
<p>Now, this is all that we need to setup our application database and the authentication part.</p>
<hr />
<h2 id="heading-setting-up-the-backup">Setting up the Backup 📥</h2>
<blockquote>
<p>ℹ️ Everything we've done up to this step is about setting up the basic project structure. In this section, we will look into the main logic of how to store user information once the user signs up for the first time in our application.</p>
</blockquote>
<p>This is how the user data backup to our database architecture is going to look like:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642589118/62199190-67e8-4838-bce5-582a176cb20e.png" alt="Project auth callback architecture" /></p>
<p>Every time a new user signs up, they are redirected to the <code>/dashboard</code> page. There, we check if the user exists in our database. If not, they are redirected to the <code>/auth/callback</code> endpoint, and the user is created in our database. If they exist, the application continues as usual.</p>
<p>In the root <code>page.tsx</code> file add these lines of code:</p>
<blockquote>
<p>ℹ️ I am using Kinde as an auth provider. The code to check user authentication will vary depending on the one you are using but the logic should be the same. If you are following along, copy and paste this code.</p>
</blockquote>
<pre><code class="lang-typescript"><span class="hljs-string">"use client"</span>;

<span class="hljs-keyword">import</span> {
  LoginLink,
  LogoutLink,
  useKindeBrowserClient,
} <span class="hljs-keyword">from</span> <span class="hljs-string">"@kinde-oss/kinde-auth-nextjs"</span>;

<span class="hljs-keyword">export</span> <span class="hljs-keyword">default</span> <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">Home</span>(<span class="hljs-params"></span>) </span>{
  <span class="hljs-keyword">const</span> { isAuthenticated } = useKindeBrowserClient();

  <span class="hljs-keyword">return</span> (
    &lt;main className=<span class="hljs-string">"flex justify-center p-24"</span>&gt;
      {!isAuthenticated ? (
        &lt;LoginLink className=<span class="hljs-string">"p-10 text-zinc-900 text-2xl font-semibold rounded-lg bg-zinc-100"</span>&gt;
          Log <span class="hljs-keyword">in</span>
        &lt;/LoginLink&gt;
      ) : (
        &lt;LogoutLink className=<span class="hljs-string">"p-10 text-zinc-900 text-2xl font-semibold rounded-lg bg-zinc-100"</span>&gt;
          Log out
        &lt;/LogoutLink&gt;
      )}
    &lt;/main&gt;
  );
}
</code></pre>
<p>This is the home page of our application, with only the <strong>login</strong> and <strong>logout</strong> buttons, based on whether the user is authenticated.</p>
<p>Once the user successfully logs in, they will be redirected to the <code>/dashboard</code> page.</p>
<p>Add the following lines of code in the <code>/app/dashboard/page.tsx</code> file:</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> { db } <span class="hljs-keyword">from</span> <span class="hljs-string">"@/db"</span>;
<span class="hljs-keyword">import</span> { getKindeServerSession } <span class="hljs-keyword">from</span> <span class="hljs-string">"@kinde-oss/kinde-auth-nextjs/server"</span>;
<span class="hljs-keyword">import</span> { redirect } <span class="hljs-keyword">from</span> <span class="hljs-string">"next/navigation"</span>;

<span class="hljs-keyword">const</span> Page = <span class="hljs-keyword">async</span> () =&gt; {
  <span class="hljs-keyword">const</span> { getUser } = getKindeServerSession();
  <span class="hljs-keyword">const</span> user = <span class="hljs-keyword">await</span> getUser();

  <span class="hljs-keyword">if</span> (!user?.id) redirect(<span class="hljs-string">"/api/auth/login"</span>);

  <span class="hljs-keyword">const</span> userInDB = <span class="hljs-keyword">await</span> db.user.findUnique({
    where: {
      id: user.id,
    },
  });

  <span class="hljs-keyword">if</span> (!userInDB) redirect(<span class="hljs-string">"/auth/callback"</span>);

  <span class="hljs-keyword">return</span> (
    &lt;div className=<span class="hljs-string">"flex flex-col justify-center items-center sm:mt-36 w-full mt-20"</span>&gt;
      &lt;h1 className=<span class="hljs-string">"font-semibold text-zinc-900 text-2xl"</span>&gt;
        You are authenticated
      &lt;/h1&gt;
      &lt;p className=<span class="hljs-string">"font-medium text-xl text-zinc-700 text-center"</span>&gt;
        The user has also been created <span class="hljs-keyword">in</span> the DB
      &lt;/p&gt;
    &lt;/div&gt;
  );
};

<span class="hljs-keyword">export</span> <span class="hljs-keyword">default</span> Page;
</code></pre>
<p>We check if the user is authenticated or not. If not, we redirect the user to the Kinde login page.</p>
<p>If they are authenticated but the user does not exist in the database, we redirect them to the <code>/auth/callback</code> endpoint where a new user is created in our database with the current logged-in user details.</p>
<p>Add the following lines of code in the <code>/src/app/auth/callback/page.tsx</code>:</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> { getKindeServerSession } <span class="hljs-keyword">from</span> <span class="hljs-string">"@kinde-oss/kinde-auth-nextjs/server"</span>;
<span class="hljs-keyword">import</span> { redirect } <span class="hljs-keyword">from</span> <span class="hljs-string">"next/navigation"</span>;
<span class="hljs-keyword">import</span> { db } <span class="hljs-keyword">from</span> <span class="hljs-string">"@/db"</span>;

<span class="hljs-keyword">const</span> Page = <span class="hljs-keyword">async</span> () =&gt; {
  <span class="hljs-keyword">const</span> { getUser } = getKindeServerSession();
  <span class="hljs-keyword">const</span> user = <span class="hljs-keyword">await</span> getUser();

  <span class="hljs-keyword">if</span> (!user?.id || !user?.email) redirect(<span class="hljs-string">"/"</span>);

  <span class="hljs-keyword">const</span> name =
    user?.given_name &amp;&amp; user?.family_name
      ? <span class="hljs-string">`<span class="hljs-subst">${user.given_name}</span> <span class="hljs-subst">${user.family_name}</span>`</span>
      : user?.given_name || <span class="hljs-literal">null</span>;

  <span class="hljs-keyword">const</span> userInDB = <span class="hljs-keyword">await</span> db.user.findFirst({
    where: {
      id: user.id,
    },
  });

  <span class="hljs-keyword">if</span> (!userInDB) {
    <span class="hljs-keyword">await</span> db.user.create({
      data: {
        id: user.id,
        email: user.email,
        ...(name &amp;&amp; { name }),
      },
    });
  }

  redirect(<span class="hljs-string">"/dashboard"</span>);
};

<span class="hljs-keyword">export</span> <span class="hljs-keyword">default</span> Page;
</code></pre>
<p>Here, we check if the user is in our database. If they exist, we redirect them to the <code>/dashboard</code> page. If the user does not exist, we create a new user in our database with their details and then redirect to the <code>/dashboard</code> page.</p>
<p>That's it! 🎉 These steps ensure that the user details in the auth provider are synced with our database and are not only stored in the auth provider.</p>
<hr />
<h2 id="heading-wrap-up"><strong>Wrap Up!</strong> ✨</h2>
<p>By now, you have a general idea of how you can backup your user information in your own database when you are using an Authentication Provider.</p>
<p>The documented source code for this article is available here:</p>
<p>https://github.com/shricodev/blogs/tree/main/auth-callback-auth-provider</p>
<p>Thank you so much for reading! 🎉 🫡</p>
<blockquote>
<p>Drop down your thoughts in the comment section below. 👇</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[✌️4 core developer tools I use in my daily life 🚀😎]]></title><description><![CDATA[This article lists my top 4 tools that I use in my daily life as a developer in 2024. ✅ These tools are aimed at improving your editing skills, terminal…]]></description><link>https://shricodev.hashnode.dev/4-core-developer-tools-i-use-in-my-daily-life</link><guid isPermaLink="true">https://shricodev.hashnode.dev/4-core-developer-tools-i-use-in-my-daily-life</guid><category><![CDATA[Productivity]]></category><category><![CDATA[Open Source]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Programming Tips]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Mon, 18 Mar 2024 04:44:42 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1710736931085/9776bc79-850c-4abe-8441-6cb6c89982ba.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR</h2>
<p>This article lists my top 4 tools that I use in my daily life as a developer in 2024. ✅</p>
<p>These tools are aimed at improving your editing skills, terminal navigation, note-taking, and utilizing Docker beyond the containerization of applications. Also, I have a small surprise for you at the end. 😉</p>
<blockquote>
<p>If you are not using at least 1-2 tools mentioned in this article, let me tell you, friend, you are missing out. Definitely give at least some of these a try. You'll thank me later. 😎</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/66ivyn1gsm2s1393lclg.gif" alt="Swag Man" /></p>
<hr />
<h2 id="heading-1-tmuxhttpsgithubcomtmuxtmuxwiki-terminal-multiplexer">1. <a target="_blank" href="https://github.com/tmux/tmux/wiki">Tmux</a> <strong>- Terminal multiplexer</strong></h2>
<blockquote>
<p>ℹ️ I don’t think there is any reason not to use Tmux. As long as you have to work in the terminal, believe me, this is going to make your life much easier.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wnmx0h067f1se6qmzkah.png" alt="Terminal Multiplexer - Tmux" /></p>
<p>Are you opening up new tabs every time you need to work on something else in the terminal, and your current terminal window is occupied? Believe me, this thing is going to blow your mind. 🤯</p>
<p>You can split a tab/window into multiple panes. Also, there is the concept of a session that allows you to have multiple windows open, completely independent of other sessions, making it easy to work on multiple projects at a time.</p>
<p>See in the image? I have my notes in another window, and dotfile configs in another. Switching between them is very easy and convenient.</p>
<p><strong>Spoiler alert</strong>: You will never want to use your mouse when working in the terminal. 😉</p>
<blockquote>
<p>It hasn't been very long since I started using Tmux, but now it's become my main core utility that I cannot live without. 🔥</p>
</blockquote>
<hr />
<h2 id="heading-2-neovimhttpsneovimio-preferred-code-editor">2. <a target="_blank" href="https://neovim.io/">Neovim</a> <strong>- Preferred Code editor</strong></h2>
<blockquote>
<p>❓ Do you love working in the terminal? If yes, then this code editor is probably what you didn’t know you needed. Give it a try.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4sk4o3tpc8lp68po6rj5.png" alt="Neovim Code Editor" /></p>
<p>I was a very big fan of VSCode, and still, I am. I knew nothing about Vim and Neovim just a few months ago. But now, believe me, in all these months, I have not touched VSCode even once. 🫠</p>
<p>Maybe you are a very big fan of VSCode as I was, but try switching yourself to Vim motions. That is the best thing you could do for yourself to increase productivity. Once you shift your editor to the terminal, you will slowly start to live in the terminal itself.</p>
<p>Although the editor in the terminal might not be to everyone's taste, at least try to use it once and see if it is something of your choice.</p>
<hr />
<h2 id="heading-3-obsidianhttpsobsidianmd-great-note-taking">3. <a target="_blank" href="https://obsidian.md">Obsidian</a> <strong>- Great Note Taking</strong></h2>
<blockquote>
<p>🧠 My second brain, and if you start using it right now, it will be yours too.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g9a5qjgzbo3515434s8j.png" alt="Obsidian Note Taking Tool" /></p>
<p>I know you might be using some cool note-taking tools such as Notion, Evernote whatever. But, do you remember the last time you opened up these note-taking apps to actually reference something you wrote a few months earlier? 🤔 See, most of you don’t have an answer. So uninstall these, and do it right now!</p>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/07t80j71gyn2txgyyica.gif" alt="Just do it GIF" /></p>
<p>This is exactly what Obsidian solves. Think of it as your <strong>second brain</strong>. This tool is so good that I have the GUI open all the time on my virtual desktop, or I open it in a Tmux window so that whenever I am writing code and need to reference my notes, I can easily do so with <strong>obsidian.nvim</strong> right from my editor. 🔥 Read more on <a target="_blank" href="https://github.com/epwalsh/obsidian.nvim">obsidian.nvim</a>.</p>
<blockquote>
<p>I also switched from Notion to Obsidian. Believe me, it was one of the best switches I made, one that I am going to cherish for the rest of my life. Don’t worry, you can import your existing notes from your note-taking tools to Obsidian pretty easily.</p>
</blockquote>
<hr />
<h2 id="heading-4-dockerhttpsdockercom-beyond-containerization">4. <a target="_blank" href="https://docker.com">Docker</a> <strong>- Beyond Containerization</strong></h2>
<blockquote>
<p>🐳 Do you use it for more than just containerizing your application?  If not, I guess it’s time to.</p>
</blockquote>
<p><img src="https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2m5r0gsamq2rbevq43hh.png" alt="Docker" /></p>
<p>Mostly when we think of Docker, we think of it just as a way to containerize applications. We know there are tons of ways to use Docker, but we simply ignore them. But, think of it more like your daily driver, not just for one purpose.</p>
<p>Recently, I wanted to try using Arch so I could say, “<strong>I use Arch, BTW!</strong>” 😉 But I didn’t want to install a completely new Linux distribution from scratch just to find out if I don’t want to move forward with Arch. For that, I simply spun up a Docker container with the Arch image and started using it. If I don’t like it in the future by any chance, I will simply remove the darn image with its container. And that’s it, I am back to normal.🔥</p>
<blockquote>
<p>⁉️ <strong>Why not use a VM for that use case?</strong></p>
</blockquote>
<p>In a VM, you have to allocate all the resources, and it will feel more bloated, to be honest. But with this approach, you have a complete, fully buttery smooth OS without having to do anything manually from scratch.</p>
<p>Also, recently I had to deal with connecting to a remote server via SSH, and my key-based authentication was not working. To debug if the problem was on my side, I simply spun up a Docker container with Alpine, set up my SSH keys there, and it connected successfully. The main problem was with the <code>ssh-daemon</code> wrong config of not accepting key-based authentication on the server itself because of <code>PubKeyAuthentication no</code> in the <code>/etc/ssh/sshd_config</code> file.</p>
<p>Docker is so beautiful 😻, try to use it very frequently.</p>
<hr />
<h2 id="heading-surprise-just-for-you">Surprise Just for You! 😉</h2>
<p>Microsoft is offering FREE Certification Courses on Cloud, DevOps, and Development! ✅</p>
<p>No payment, no subscription, and no registration is required. Just start learning! 🚀</p>
<blockquote>
<p>⚠️ <strong>NOTE</strong>: You will be redirected to the Official Microsoft Website.</p>
</blockquote>
<p>https://learn.microsoft.com/training?wt.mc_id=studentamb_366508</p>
<p>Thank you for reading! I hope you try out some of these at least. 🫡</p>
]]></content:encoded></item><item><title><![CDATA[😻Build your own CLI version of MonkeyType 🙈]]></title><description><![CDATA[In this easy-to-follow tutorial, you will learn how to build your own CLI version of MonkeyType in minutes. 😎 What you will learn: ✨ Are you ready to…]]></description><link>https://shricodev.hashnode.dev/build-your-own-cli-version-of-monkeytype</link><guid isPermaLink="true">https://shricodev.hashnode.dev/build-your-own-cli-version-of-monkeytype</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Sun, 18 Feb 2024 13:13:24 GMT</pubDate><enclosure url="https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F84yzvrzheoarbc30l0jk.gif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR</h2>
<p>In this easy-to-follow tutorial, you will learn how to build your own CLI version of MonkeyType in minutes. 😎</p>
<p><strong>What you will learn: ✨</strong></p>
<ul>
<li>Use the Python curses module to build a robust typing CLI application with <strong>WPM</strong> and <strong>Accuracy</strong> support.</li>
</ul>
<p>Are you ready to become a CLI MonkeyTyper? 😉 Whether this is your first CLI application or nth application. Feel free to follow along.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642544577/8680d70a-b032-46eb-853e-4ef41a305b1a.gif" alt="Monkey typing on a Laptop" /></p>
<hr />
<h2 id="heading-setting-up-the-environment">Setting up the Environment 🙈</h2>
<blockquote>
<p>ℹ️ There is no need to set up a virtual environment, we will not be using any external dependencies.</p>
</blockquote>
<p>Create a folder to keep all your source code for the project:</p>
<pre><code class="lang-bash">mkdir cli-monkeytype-python
<span class="hljs-built_in">cd</span> cli-monkeytype-python
</code></pre>
<p>Create two new files, where we will code the program:</p>
<pre><code class="lang-bash">touch main.py typing_test.py
</code></pre>
<p>The <code>main.py</code> file will act as the starting point of our application and the <code>typing_test.py</code> file will hold all the logic of the program.</p>
<blockquote>
<p>ℹ️ For Linux or Mac users, you don't need to download any dependencies, we will mainly be using <strong>curses</strong>, <strong>time</strong>, and <strong>random</strong> modules which are all included in the Python Standard Library.</p>
</blockquote>
<p>⚠️ <strong>Note</strong></p>
<blockquote>
<p>Windows users might have to install curses as it is not included in the Python Standard Library for Windows. Make sure to have it installed before proceeding further.</p>
</blockquote>
<hr />
<h2 id="heading-lets-code-it-up">Let's Code it up 🐵</h2>
<blockquote>
<p>💡 We will look into the Approach, Outline, and the actual coding portion of the application in this section. 😵‍💫</p>
</blockquote>
<h3 id="heading-approach-and-outline">Approach and Outline 👀</h3>
<p>We will be taking a different approach here, instead of jamming all the code in the <code>main</code> file. We will split the codes into classes in a different file.</p>
<p>There will be a separate file containing a class responsible for encapsulating all the logic related to the Typing Test. In the main file, we will then invoke the methods from this class. Sounds, right? Let's get into it. 🚀</p>
<p>Here, is the skeleton of our class and all the methods that we are going to be working on.</p>
<pre><code class="lang-python"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">TypingTest</span>:</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, stdscr</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_line_to_type</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_wpm</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_accuracy</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_typed_chars</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_details</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">test_accuracy</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">test_wpm</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">pass</span>
</code></pre>
<p>All the function names should be self-explanatory. If you need help understanding what each function does, even after looking at this outline, why are you even reading the article? Just kidding *not really*. 😏</p>
<blockquote>
<p>🥱 This is a beginner-friendly application. Don't worry, code along.</p>
</blockquote>
<h3 id="heading-actual-fun-begins">Actual Fun Begins!</h3>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642547118/40cd7878-4a73-4c3e-9c63-19a99342712c.gif" alt="Showtime GIF" /></p>
<p>We will start by importing modules and coding our <code>__init__</code> method. This will initialize all the jargon we need for the program to work.</p>
<pre><code class="lang-Python"><span class="hljs-keyword">import</span> curses
<span class="hljs-keyword">import</span> random
<span class="hljs-keyword">import</span> time

<span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">TypingTest</span>:</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">__init__</span>(<span class="hljs-params">self, stdscr</span>):</span>
        self.stdscr = stdscr
        self.to_type_text = self.get_line_to_type()
        self.user_typed_text = []
        self.wpm = <span class="hljs-number">0</span>
        self.start_time = time.time()

        <span class="hljs-comment"># Initialize color pairs</span>
        curses.init_pair(<span class="hljs-number">3</span>, curses.COLOR_WHITE, curses.COLOR_BLACK)
        curses.init_pair(<span class="hljs-number">2</span>, curses.COLOR_RED, curses.COLOR_BLACK)
        curses.init_pair(<span class="hljs-number">1</span>, curses.COLOR_GREEN, curses.COLOR_BLACK)
    <span class="hljs-comment"># --SNIP--</span>
</code></pre>
<p>The <code>stdscr</code> is used to control the terminal screen and is essential for creating text-based user interfaces where users can see their keystrokes. ⌨️</p>
<p>The <code>get_line_to_type</code> method gets a line of text for the user to type. That text is stored in the <code>self.to_type_text</code> variable. As they type, the characters they enter are saved in the <code>self.user_typed_text</code> list. We use a list because it will be easier to pop the last item when the user corrects their mistyped character.</p>
<p>The initial words per minute (WPM) score is set to 0, and we record the start time of the test. We also initialize a few <strong>color pairs</strong> that we will use to indicate colors on the characters based on whether they are correct or not. Later, we’ll calculate the <strong>WPM</strong> based on how long it takes the user to type.</p>
<p>Now, add the code for the following functions</p>
<blockquote>
<p>ℹ️ Make sure to create a new file named <code>typing_texts.txt</code> in the project root with a few lines of text. For reference: <a target="_blank" href="https://github.com/shricodev/blogs/blob/main/cli-monkeytype-python/typing_texts.txt">click here</a>.</p>
</blockquote>
<pre><code class="lang-Python">    <span class="hljs-comment"># --SNIP--</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">get_line_to_type</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">with</span> open(<span class="hljs-string">"typing_texts.txt"</span>, <span class="hljs-string">"r"</span>, encoding=<span class="hljs-string">"utf-8"</span>) <span class="hljs-keyword">as</span> file:
            lines = file.readlines()

        <span class="hljs-keyword">return</span> random.choice(lines).strip()

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_wpm</span>(<span class="hljs-params">self</span>):</span>
        self.stdscr.addstr(<span class="hljs-number">1</span>, <span class="hljs-number">0</span>, <span class="hljs-string">f"WPM: <span class="hljs-subst">{self.wpm}</span>"</span>, curses.color_pair(<span class="hljs-number">3</span>))

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_accuracy</span>(<span class="hljs-params">self</span>):</span>
        self.stdscr.addstr(
            <span class="hljs-number">2</span>,
            <span class="hljs-number">0</span>,
            <span class="hljs-string">f"Accuracy: <span class="hljs-subst">{self.test_accuracy()}</span>%"</span>,
            curses.color_pair(<span class="hljs-number">3</span>),
        )

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_typed_chars</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-keyword">for</span> i, char <span class="hljs-keyword">in</span> enumerate(self.user_typed_text):
            correct_character = self.to_type_text[i]
            <span class="hljs-comment"># Use color pair 1 if correct, else color pair 2.</span>
            color = <span class="hljs-number">1</span> <span class="hljs-keyword">if</span> char == correct_character <span class="hljs-keyword">else</span> <span class="hljs-number">2</span>
            self.stdscr.addstr(<span class="hljs-number">0</span>, i, char, curses.color_pair(color))

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">display_details</span>(<span class="hljs-params">self</span>):</span>
        self.stdscr.addstr(self.to_type_text)
        self.display_wpm()
        self.display_accuracy()
        self.display_typed_chars()
    <span class="hljs-comment"># --SNIP--</span>
</code></pre>
<p>Let me summarise these methods, they are pretty straightforward:</p>
<p>🎯 <code>get_line_to_type(self)</code>: Retrieves a random line from a file named "typing_texts.txt" with removed trailing spaces.</p>
<p>🎯 <code>display_wpm(self)</code>: Displays the WPM on the screen as the user types, in the first row.</p>
<p>🎯 <code>display_accuracy(self)</code>: Displays the <strong>accuracy percentage</strong> on the screen at row 2. The accuracy is calculated by the <code>test_accuracy()</code> method which we will write soon.</p>
<p>🎯 <code>display_typed_chars(self)</code>: Displays the characters the user has typed on the screen, highlighting correct characters in one color pair (color 1) and incorrect characters in another color pair (color 2).</p>
<p>🎯 <code>display_details(self)</code>: It is essentially a helper function that helps display contents from all the display functions above.</p>
<p>Okay, now that we have written these display methods, let's implement the actual logic to test the accuracy and the WPM itself.</p>
<p>Add the following lines of code:</p>
<pre><code class="lang-Python">    <span class="hljs-comment"># --SNIP--</span>
    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">test_accuracy</span>(<span class="hljs-params">self</span>):</span>
        total_characters = min(len(self.user_typed_text), len(self.to_type_text))

        <span class="hljs-comment"># If there are no typed chars, show accuracy 0.</span>
        <span class="hljs-keyword">if</span> total_characters == <span class="hljs-number">0</span>:
            <span class="hljs-keyword">return</span> <span class="hljs-number">0.0</span>

        matching_characters = <span class="hljs-number">0</span>

        <span class="hljs-keyword">for</span> current_char, target_char <span class="hljs-keyword">in</span> zip(self.user_typed_text, self.to_type_text):
            <span class="hljs-keyword">if</span> current_char == target_char:
                matching_characters += <span class="hljs-number">1</span>

        matching_percentage = (matching_characters / total_characters) * <span class="hljs-number">100</span>
        <span class="hljs-keyword">return</span> matching_percentage

    <span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">test_wpm</span>(<span class="hljs-params">self</span>):</span>
        <span class="hljs-comment"># getkey method by default is blocking.</span>
        <span class="hljs-comment"># We do not want to wait until the user types each char to check WPM.</span>
        <span class="hljs-comment"># Else the entire logic will be faulty.</span>
        self.stdscr.nodelay(<span class="hljs-literal">True</span>)

        <span class="hljs-keyword">while</span> <span class="hljs-literal">True</span>:
            <span class="hljs-comment"># Since we have nodelay = True, if not using max(), </span>
            <span class="hljs-comment"># users might end up with time.time() equal to start_time,</span>
            <span class="hljs-comment"># resulting in 0 and potentially causing a zero-divisible error in the below line.</span>
            time_elapsed = max(time.time() - self.start_time, <span class="hljs-number">1</span>)

            <span class="hljs-comment"># Considering the average word length in English is 5 characters</span>
            self.wpm = round((len(self.user_typed_text) / (time_elapsed / <span class="hljs-number">60</span>)) / <span class="hljs-number">5</span>)
            self.stdscr.clear()
            self.display_details()
            self.stdscr.refresh()

            <span class="hljs-comment"># Exit the loop when the user types in the total length of the text.</span>
            <span class="hljs-keyword">if</span> len(self.user_typed_text) == len(self.to_type_text):
                self.stdscr.nodelay(<span class="hljs-literal">False</span>)
                <span class="hljs-keyword">break</span>

            <span class="hljs-comment"># We have `nodelay = True`, so we don't want to wait for the keystroke.</span>
            <span class="hljs-comment"># If we do not get a key, it will throw an exception</span>
            <span class="hljs-comment"># in the below lines when accessing the key.</span>
            <span class="hljs-keyword">try</span>:
                key = self.stdscr.getkey()
            <span class="hljs-keyword">except</span> Exception:
                <span class="hljs-keyword">continue</span>

            <span class="hljs-comment"># Check if the key is a single character before using ord()</span>
            <span class="hljs-keyword">if</span> isinstance(key, str) <span class="hljs-keyword">and</span> len(key) == <span class="hljs-number">1</span>:
                <span class="hljs-keyword">if</span> ord(key) == <span class="hljs-number">27</span>:  <span class="hljs-comment"># ASCII value for ESC</span>
                    <span class="hljs-keyword">break</span>

            <span class="hljs-comment"># If the user has not typed anything reset to the current time</span>
            <span class="hljs-keyword">if</span> <span class="hljs-keyword">not</span> self.user_typed_text:
                self.start_time = time.time()

            <span class="hljs-keyword">if</span> key <span class="hljs-keyword">in</span> (<span class="hljs-string">"KEY_BACKSPACE"</span>, <span class="hljs-string">"\b"</span>, <span class="hljs-string">"\x7f"</span>):
                <span class="hljs-keyword">if</span> len(self.user_typed_text) &gt; <span class="hljs-number">0</span>:
                    self.user_typed_text.pop()

            <span class="hljs-keyword">elif</span> len(self.user_typed_text) &lt; len(self.to_type_text):
                self.user_typed_text.append(key)
</code></pre>
<p>🎯 <code>test_accuracy(self)</code>: Calculates and returns the typing accuracy as a percentage by comparing the characters typed by the user with the target text. If the character matches, it increases the count of matching characters by 1. In the end, it calculates the percentage of the match.</p>
<p>🎯 <code>test_wpm(self)</code>: Calculates the words per minute (WPM) and updates the display in real time. We use a <strong>formula</strong> to calculate the WPM, it is not something I came up with, I copied it from the internet. It tracks what the user types, handles backspaces, and stops when they finish typing the target text or press <strong>ESC</strong>.</p>
<p>Great! This is it for our <strong>TypingTest</strong> class. 🎉</p>
<blockquote>
<p>✅ We have written the code in such a way that it will help us easily import this code into any future projects and make maintenance a lot easier.</p>
</blockquote>
<p>Time to test our implementation. 🙈</p>
<p>In the <code>main.py</code> file, add the following lines of code:</p>
<pre><code class="lang-Python"><span class="hljs-keyword">from</span> curses <span class="hljs-keyword">import</span> wrapper
<span class="hljs-keyword">from</span> typing_test <span class="hljs-keyword">import</span> TypingTest

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">main</span>(<span class="hljs-params">stdscr</span>):</span>
    stdscr.clear()
    stdscr.addstr(<span class="hljs-string">"Welcome to the typing speed test"</span>)
    stdscr.addstr(<span class="hljs-string">"\nPress any key to continue!"</span>)

    <span class="hljs-keyword">while</span> <span class="hljs-literal">True</span>:
        typing_test = TypingTest(stdscr)
        stdscr.getkey()
        typing_test.test_wpm()
        stdscr.addstr(
            <span class="hljs-number">3</span>,
            <span class="hljs-number">0</span>,
            <span class="hljs-string">"Congratulations! You have completed the test! Press any key to continue..."</span>,
        )
        stdscr.nodelay(<span class="hljs-literal">False</span>)
        key = stdscr.getkey()

        <span class="hljs-comment"># Check if the key is a single character before using ord()</span>
        <span class="hljs-keyword">if</span> isinstance(key, str) <span class="hljs-keyword">and</span> len(key) == <span class="hljs-number">1</span>:
            <span class="hljs-keyword">if</span> ord(key) == <span class="hljs-number">27</span>:  <span class="hljs-comment"># ASCII value for ESC</span>
                <span class="hljs-keyword">break</span>

<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    wrapper(main)
</code></pre>
<blockquote>
<p>💡 NOTE: we are calling the main function inside the <code>wrapper</code> method from curses which handles initialization and cleanup of the curses module.</p>
</blockquote>
<p>Inside main, we make an instance of the <strong>TypingTest</strong> class and run the test in the infinite loop, which lets the user keep on running the tests until they decide to quit by pressing <strong>ESC</strong>.</p>
<p>Let's see it in action. 🔥</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642550029/587b1ea1-b93c-427e-a4f7-aeb6570fe0ec.gif" alt="Typing Test Demo" /></p>
<blockquote>
<p>🫵 If you have made it this far, I want to assign you a small task. Currently, we are randomly selecting text from a file for typing. I would like you to <strong>scrape</strong> typing text from the internet and use that content instead. Feel free to open a pull request in my repository with your changes.</p>
<p>If you need help, I have already worked on a similar Python scraping project. Feel free to <a target="_blank" href="https://github.com/shricodev/IPO-Monitor-NEPSE">check</a> it out.</p>
</blockquote>
<hr />
<h2 id="heading-wrap-up"><strong>Wrap Up!</strong> 🐒</h2>
<p>By now, you have built a Python CLI application to test your typing speed right in your terminal.</p>
<p>The documented source code for this article is available here:</p>
<p>https://github.com/shricodev/blogs/tree/main/cli-monkeytype-python</p>
<p>Thank you so much for reading! 🎉🫡</p>
<blockquote>
<p>Drop down your thoughts in the comment section below. 👇</p>
</blockquote>
]]></content:encoded></item><item><title><![CDATA[✨ Top 5 Underrated Open Source Projects that no one talks about 🫵🤐]]></title><description><![CDATA[This article lists five great projects that are not very popular that you should give a shot. 🔥 These tools are aimed at improving data processing, API…]]></description><link>https://shricodev.hashnode.dev/top-5-underrated-open-source-projects-that-no-one-talks-about</link><guid isPermaLink="true">https://shricodev.hashnode.dev/top-5-underrated-open-source-projects-that-no-one-talks-about</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Thu, 11 Jan 2024 15:12:26 GMT</pubDate><enclosure url="https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Frj12gooh0fc8fdpzztxl.gif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR</h2>
<p>This article lists five great projects that are not very popular that you should give a shot. 🔥</p>
<p>These tools are aimed at improving <strong>data processing</strong>, <strong>API development</strong>, <strong>backend testing</strong>, <strong>authentication</strong>, and <strong>secure tunneling</strong>.</p>
<p>Open-source projects such as these rely on community support 🙏, so consider exploring and starring these repositories to contribute to their growth.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642429197/a4e18d48-87ce-447b-a183-45a0148349c9.gif" alt="Hug GIF" /></p>
<hr />
<h2 id="heading-1-esprochttpscudatacom-data-processing">1. <a target="_blank" href="http://scudata.com">esProc</a> <strong>- Data Processing</strong></h2>
<blockquote>
<p>💡 esProc is a scripting language for data processing with rich library functions and powerful syntax.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642431111/e2a3a574-6ca7-4656-8abe-8e51ee0311df.jpeg" alt="esProc scripting language for data processing" /></p>
<p>esProc is a computing and processing engine for structured and semi-structured data. esProc is neither an SQL system nor NoSQL technology (such as MongoDB), and instead, it adopts self-created SPL (Structured Process Language) syntax, which is simpler in coding and creating highly efficient programs with existing data processing technologies.</p>
<p>esProc is a <strong>pure Java</strong> written that makes it easy to add powerful data processing capabilities to your Java 🍵 applications but non-Java applications can invoke esProc through RESTful APIs.</p>
<h3 id="heading-popular-faqs">Popular FAQs 🤔</h3>
<blockquote>
<p><strong>⬇️ What platforms can esProc run?</strong></p>
</blockquote>
<p>As it is built purely in Java, it can run smoothly in any OS equipped with JVM (Java Virtual Machine), cloud servers, or even containers. 😎</p>
<blockquote>
<p><strong>⬇️ Can esProc run based on the existing database?</strong></p>
</blockquote>
<p>Yes, of course! esProc supports dozens of data sources, including databases, text, excel, json/xml, web service, etc.</p>
<blockquote>
<p><strong>⬇️ Why ditch SQL for esProc?</strong></p>
</blockquote>
<p>Simplified stepwise code, easy to write and debug. Reduce the development, hardware, operation, and maintenance costs compared to SQL by N times.</p>
<blockquote>
<p>🟢 I have recently written an article on this tool, highlighting its powerful features. Check it <a target="_blank" href="https://dev.to/shricodev/one-must-have-tool-for-anyone-in-data-field-2jek">out.</a> </p>
<p>If you want to dive deeper into the potential of this tool, <a target="_blank" href="https://dev.to/jbx1279"><strong>jbx1279</strong></a> shares some insightful articles on esProc and SPL itself. Make sure to check them out too.</p>
</blockquote>
<hr />
<h2 id="heading-2-firecamphttpsfirecampdev-postman-alternative">2. <a target="_blank" href="https://firecamp.dev/">Firecamp</a> <strong>- Postman alternative</strong></h2>
<blockquote>
<p>💡 API development platform that helps developers design, develop, test, and document their APIs effortlessly.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642432937/882be3b0-b844-461d-99d5-700edacda5bf.jpeg" alt="Firecamp tool Postman alternative" /></p>
<p>Firecamp is open open-source Postman alternative with VScode DX, an API development platform prioritizing developer experience, and offering a seamless environment for designing, testing, and documenting APIs. 🎯</p>
<p>With Firecamp, collaborate on API collections across the workspace and team and build APIs faster without switching between tools and apps. Documentation, CLI, CI/CD under one roof.</p>
<blockquote>
<p><strong>⬇️ Is it challenging for me to switch from Postman to Firecamp?</strong></p>
</blockquote>
<p>You can seamlessly transfer your Postman scripts and data like <strong>API Collection</strong>, and <strong>environment variables</strong> to Firecamp with no issues at all.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642435037/00407554-f25d-49a0-ba51-30b4a985f2ce.png" alt="Firecamp Postman alternative" /></p>
<hr />
<h2 id="heading-3-keployhttpskeployio-backend-testing">3. <a target="_blank" href="https://keploy.io/">Keploy</a> <strong>- Backend Testing</strong></h2>
<blockquote>
<p>💡 Generates tests and stubs for your application that actually work!</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642439088/ce26b944-b099-4488-b2da-d3f89d0986d1.jpeg" alt="Keploy Generate Backend Tests" /></p>
<p>Keploy is your open-source, developer-centric backend testing tool. It makes backend testing easy and productive for engineering teams. With Keploy we don't have to write manual test cases.</p>
<p>It records API interactions and expected responses and generates test cases and data mocks to make our work easy and efficient significantly speeding up releases and enhancing reliability. 📈</p>
<blockquote>
<p><strong>⬇️ Is it a unit testing framework? or does it replace unit tests entirely?</strong></p>
</blockquote>
<p>Keploy goes well with unit testing frameworks like <code>go-test</code>, <code>Pytest</code>, or <code>Jest</code>, smoothing the testing processes and saving up to 80% effort. While it covers most cases, you may still choose to write tests for non-API-invokable methods.</p>
<blockquote>
<p><strong>⬇️ Do I need to do code changes to integrate Keploy in my application?</strong></p>
</blockquote>
<p>No. Keploy plays nice with your existing codebase, requiring no code changes.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642441162/0c499f6d-2aeb-4f8c-91be-7cd9bb9a7e2e.gif" alt="Keploy Backend Testing Demo" /></p>
<hr />
<h2 id="heading-4-hankohttpshankoio-passkeys-authentication">4. <a target="_blank" href="https://hanko.io">Hanko</a> <strong>- Passkeys Authentication</strong></h2>
<blockquote>
<p>💡 Passwordless authentication server supporting FIDO2 and WebAuthn standards.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642443645/9f00095a-fd90-4b5a-858d-062510702ce5.jpeg" alt="Hanko Passkeys Authentication" /></p>
<p>Hanko is a lightweight, open-source user authentication solution that takes you on the journey beyond passwords. It supports FIDO2 and WebAuthn standards, providing a secure and seamless user authentication experience.</p>
<blockquote>
<p><strong>⬇️ How does Hanko work?</strong></p>
</blockquote>
<p>Hanko works by registering and authenticating users with their own devices, such as smartphones, laptops, or security keys. These devices act as cryptographic tokens that prove the user’s identity without requiring passwords or other credentials. Hanko also supports various authentication methods, such as biometrics in mobile app or OAuth login.</p>
<blockquote>
<p><strong>⬇️ How do I get started with Hanko?</strong></p>
</blockquote>
<p>You can get started with Hanko by signing up for a free account and following the documentation and tutorials. For production use, go for Hanko Cloud.</p>
<blockquote>
<p>🟢 I have recently built a project using Hanko Passkeys authentication. Check it out <a target="_blank" href="https://github.com/shricodev/pdfwhisper-openai"><strong>here</strong></a>.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642445228/7e64efb4-b21c-4d02-8292-f870c99c286c.png" alt="Hanko Landing Page" /></p>
<hr />
<h2 id="heading-5-zrokhttpszrokio-ngrok-on-steroids">5. <a target="_blank" href="https://zrok.io/">Zrok</a> <strong>- Ngrok on Steroids</strong></h2>
<blockquote>
<p>💡 Alternative to Ngrok, offering enhanced features and a free SaaS model.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642447050/4d711d9f-d251-45fa-b320-a9226d36bcb2.jpeg" alt="Zrok ngrok alternative" /></p>
<p>Zrok is a tool that is built on top of <strong>OpenZiti</strong> and helps share a running service, like a web server or a network socket, or to share a directory of static files to the internet securely. It is an alternative to Ngrok but with some enhanced features and a <strong>free SaaS</strong> model.</p>
<p>With Zrok, you get the ability to create secure tunnels for your applications, making it easier to share and collaborate on your projects.</p>
<blockquote>
<p><strong>⬇️ What are the benefits of using Zrok over Ngrok?</strong></p>
</blockquote>
<p>Zrok has a built-in authentication system, a web dashboard to manage tunnels, and a free SaaS model. It is also completely <strong>self-hostable</strong>.</p>
<blockquote>
<p><strong>⬇️ How do I get started with Zrok?</strong></p>
</blockquote>
<p>To get started with Zrok, Download the Zrok client for your platform or use the web interface to create a tunnel. You can also use the Zrok CLI to create and manage your tunnels from the command line.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642449032/d2ae3a1e-1be6-46a5-b38d-de02f5049229.png" alt="Zrok secure tunnelling" /></p>
<hr />
<blockquote>
<p>If you think of any other handy projects you use that are not as popular as they should be, please share them in the comments section below. 👇</p>
</blockquote>
<p>Thank you so much for reading! 🎉🫡</p>
]]></content:encoded></item><item><title><![CDATA[🤯 How easy is it to integrate an AI chatbot into your app in 2024? 🏆🥳]]></title><description><![CDATA[In this tutorial, you will learn how to add AI chatbot support to your Python application in minutes. Explore OpenAI and learn how to easily integrate it…]]></description><link>https://shricodev.hashnode.dev/how-easy-is-it-to-integrate-an-ai-chatbot-into-your-app-in-2024</link><guid isPermaLink="true">https://shricodev.hashnode.dev/how-easy-is-it-to-integrate-an-ai-chatbot-into-your-app-in-2024</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Tue, 02 Jan 2024 11:28:40 GMT</pubDate><enclosure url="https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fryf9ssngpt7mj5j56yn1.gif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr"><strong>TL;DR</strong></h2>
<p>In this tutorial, you will learn how to add <strong>AI chatbot</strong> support to your Python application in minutes.</p>
<p>Explore <strong>OpenAI</strong> and learn how to easily integrate it into your existing application with the <a target="_blank" href="https://pypi.org/project/openai/">OpenAI</a> Python module.</p>
<p><strong>On the agenda 🔥:</strong></p>
<ul>
<li>How to use the OpenAI GPT model to generate chat responses in real time.</li>
</ul>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642398580/09e1be5c-72f6-40d9-91c4-7742f26afbb2.gif" alt="Buckle Up" /></p>
<hr />
<h2 id="heading-lets-code-it-up"><strong>Let's code it up</strong> 🧙🏻‍♂️</h2>
<blockquote>
<p>✨ For this tutorial, we will build a Python application from scratch with OpenAI.</p>
</blockquote>
<h3 id="heading-set-up-the-environment"><strong>Set up the environment</strong> 🔥</h3>
<p>Create a folder for the project:</p>
<pre><code class="lang-bash">mkdir easy-build-an-ai-2024
<span class="hljs-built_in">cd</span> easy-build-an-ai-2024
</code></pre>
<p>Now, let's create a virtual environment for our Python application. Run this command to create a virtual environment using <code>venv</code>:</p>
<pre><code class="lang-bash">python -m venv venv
</code></pre>
<p>Activate the virtual environment on Linux:</p>
<pre><code class="lang-bash"><span class="hljs-built_in">source</span> venv/bin/activate
</code></pre>
<blockquote>
<p>ℹ️ Most of us use the fish shell, to activate the environment on the fish shell, run the following command:</p>
</blockquote>
<pre><code class="lang-bash">. venv/bin/activate.fish
</code></pre>
<blockquote>
<p>ℹ️ For Windows users, run the following command:</p>
</blockquote>
<pre><code class="lang-bash">venv/bin/activate
</code></pre>
<p>After running the command, the virtual environment should be activated.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642400866/ba7ffd70-5855-4f18-a5e6-1ce3c4c8a4f5.png" alt="Python Virtual Environment" /></p>
<p>Now, install all the necessary packages required for the project:</p>
<pre><code class="lang-bash">pip install openai python-dotenv
</code></pre>
<blockquote>
<p>💡 To communicate with OpenAI, we will need an API key. First, create an account on OpenAI and obtain the API key.</p>
</blockquote>
<p>In the project's root, create a new file named <code>.env</code>, add a new environment variable called <code>OPENAI_API_KEY</code>, and paste the API key you just generated.</p>
<pre><code class="lang-plaintext"># 👇 /.env
OPENAI_API_KEY=&lt;YOUR_API_KEY&gt;
</code></pre>
<h3 id="heading-lets-code"><strong>Let's Code</strong> 👨‍💻</h3>
<p>Create a new file called <code>main.py</code> in the project's root. Add the following chunk of code:</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 /main.py</span>
<span class="hljs-keyword">import</span> os

<span class="hljs-keyword">import</span> openai
<span class="hljs-keyword">from</span> dotenv <span class="hljs-keyword">import</span> find_dotenv, load_dotenv

<span class="hljs-comment"># find .env file</span>
dotenv_path = find_dotenv()

<span class="hljs-comment"># load up the entries as environment variables</span>
load_dotenv(dotenv_path)

<span class="hljs-comment"># get the OPENAI_API_KEY from environment variable</span>
OPENAI_API_KEY = os.getenv(<span class="hljs-string">"OPENAI_API_KEY"</span>)

<span class="hljs-comment"># set the api key for openai</span>
openai.api_key = OPENAI_API_KEY
</code></pre>
<p>We are importing the required modules, retrieving the environment variables from the <code>.env</code> file, and setting up the OpenAI API key with the one we just created.</p>
<p>Now, we need to set up the actual code that talks with the OpenAI.</p>
<pre><code class="lang-python"><span class="hljs-comment"># 👇 /main.py</span>
...
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">chat_with_openai</span>(<span class="hljs-params">prompt</span>):</span>
    <span class="hljs-keyword">try</span>:
        <span class="hljs-comment"># Create a chat completion with OpenAI API</span>
        response = openai.chat.completions.create(
            model=<span class="hljs-string">"gpt-3.5-turbo"</span>,
            messages=[{<span class="hljs-string">"role"</span>: <span class="hljs-string">"user"</span>, <span class="hljs-string">"content"</span>: prompt}],
        )

        <span class="hljs-comment"># Extract the content of the message from the response</span>
        response_content = response.choices[<span class="hljs-number">0</span>].message.content

        <span class="hljs-comment"># Strip any leading or trailing whitespace from the response content</span>
        response_content_stripped = response_content.strip()

        <span class="hljs-keyword">return</span> response_content_stripped

    <span class="hljs-keyword">except</span> Exception <span class="hljs-keyword">as</span> e:
        print(<span class="hljs-string">f"An error occurred: <span class="hljs-subst">{str(e)}</span>"</span>)
        <span class="hljs-keyword">return</span> <span class="hljs-string">"Sorry, something went wrong. Please try again."</span>


<span class="hljs-keyword">if</span> __name__ == <span class="hljs-string">"__main__"</span>:
    <span class="hljs-keyword">while</span> <span class="hljs-literal">True</span>:
        user_input = input(<span class="hljs-string">"ASK ANYTHING &gt;&gt; "</span>)

        <span class="hljs-keyword">if</span> user_input.lower() <span class="hljs-keyword">in</span> [<span class="hljs-string">"bye"</span>, <span class="hljs-string">"quit"</span>, <span class="hljs-string">"exit"</span>]:
            <span class="hljs-keyword">break</span>

        <span class="hljs-comment"># Get a response from the OpenAI model</span>
        response = chat_with_openai(user_input)
        print(<span class="hljs-string">"RESPONSE: "</span>, response)
</code></pre>
<p>The code uses OpenAI's <code>GPT-3.5 Turbo</code> model to generate responses, and it runs a continuous loop for user input until the user decides to exit.</p>
<p>And, this is it. Our simple Python application with OpenAI chatbot support is all set. 🎉</p>
<p>Run the application in the terminal and talk with GPT.</p>
<pre><code class="lang-bash">python main.py
</code></pre>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642402285/ea6a705e-e87d-4e77-b1cc-9ac0bd114253.png" alt="Python application terminal demo" /></p>
<hr />
<h2 id="heading-wrap-up"><strong>Wrap Up!</strong> ✨</h2>
<p>By now, you have a generic understanding of how easy it is to integrate an AI chatbot into your application just with the help of a few functions from OpenAI in your Python application.</p>
<blockquote>
<p>Please let me know your thoughts on the article in the comments section below. 👇</p>
</blockquote>
<p>The source code for this article is available here:</p>
<p>https://github.com/shricodev/blogs/tree/main/easy-build-an-ai-2024</p>
<p>So, that is it for this article. Thank you so much for reading! 🎉🫡</p>
]]></content:encoded></item><item><title><![CDATA[🎉One must-have tool for anyone in Data field 🤯✨]]></title><description><![CDATA[SQL databases are great for handling structured data and complex queries, but they fall short when performing complex calculations and optimizing…]]></description><link>https://shricodev.hashnode.dev/one-must-have-tool-for-anyone-in-data-field</link><guid isPermaLink="true">https://shricodev.hashnode.dev/one-must-have-tool-for-anyone-in-data-field</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Mon, 11 Dec 2023 12:25:38 GMT</pubDate><enclosure url="https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fhglr44cm530vq1m8y71y.gif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr"><strong>TL;DR</strong></h2>
<p>SQL databases are great for handling structured data and complex queries, but they fall short when performing complex calculations and optimizing performance. 🐢</p>
<p>I am not sure if you are ready to read this. 🤐 I think most of you might be shocked by reading this.</p>
<blockquote>
<p>✨ <strong>There is a faster and probably better alternative to SQL</strong> 😮</p>
</blockquote>
<p>I recently came across this, and let me tell you one thing: '<strong>It is really good!</strong>' Let me introduce you to <strong>esProc</strong>. 🏃‍♂️💨</p>
<p><a target="_blank" href="https://www.scudata.com/"><strong>esProc SPL</strong></a> is an open-source next-gen data processing language that can surpass the limitations of SQL and boost your data processing. 🤩</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642342087/0631cca3-2771-413f-a457-3b808bab99d3.gif" alt="Buckle Up GIF" /></p>
<hr />
<h2 id="heading-what-even-is-spl"><strong>What even is SPL?</strong> 🤔</h2>
<blockquote>
<p>💡 SPL short for <strong>Structured Process Language</strong> is software for computing structured/semi-structured data.</p>
</blockquote>
<p>I am sure many of you may not know about SPL. There is just a letter's difference between <strong>SQL</strong> and <strong>SPL</strong>, yet they vary in many ways. 😵‍💫</p>
<p>SPL uses a grid-based 📊 syntax instead of text, making it more visual and concise. As a data computing engine, SPL can achieve higher performance 💨 compared to traditional SQL.</p>
<p>SPL is specially dedicated to low code 🧑‍💻 and high performance. At its core, it is software comprising three major pieces:</p>
<ul>
<li><strong>Programming Language 🚀</strong>: SPL is a language that facilitates the process of structured data computing, and has excellent performance in handling most computing scenarios.</li>
</ul>
<p><br /></p>
<ul>
<li><strong>Data Computing Middleware 🤝</strong>: SPL provides independent computation ability independent of databases. It can perform various structured data computations on its own and is easily embeddable in applications. 📱</li>
</ul>
<p><br /></p>
<ul>
<li><strong>Big Data Computing Platform 📈</strong>: SPL has its cluster computing system for large data, providing a distributed environment for programmers to control task distribution and implement efficient algorithms. 🧠</li>
</ul>
<blockquote>
<p>👉 To learn more about SPL, follow this <a target="_blank" href="https://blog.scudata.com/a-programming-language-coding-in-a-grid/"><strong>link</strong></a>.</p>
</blockquote>
<hr />
<h2 id="heading-how-spl-can-be-an-alternative-to-sql"><strong>How SPL can be an alternative to SQL?</strong> 🧐</h2>
<p>Before talking about whether SPL can be an alternative to SQL, let's first take a look into some limitations of SQL.</p>
<ul>
<li><strong>SQL queries grow in complexity fairly quickly</strong> 🧵</li>
</ul>
<p>SQL is considered a pretty easy language to write. It often reads like plain English but it is not always the case. 😓</p>
<p>When writing nested queries, it grows in complexity and even a senior developer 🧑‍💻 finds it hard to make sense of it.</p>
<p>Here's a simple SQL query to calculate the maximum number of consecutive days that a stock price keeps rising. See how quickly the SQL statement is harder to read. 😵‍💫</p>
<pre><code class="lang-SQL"><span class="hljs-keyword">SELECT</span> <span class="hljs-keyword">MAX</span>(consecutive_day)
<span class="hljs-keyword">FROM</span> (
    <span class="hljs-keyword">SELECT</span> <span class="hljs-keyword">COUNT</span>(*) <span class="hljs-keyword">as</span> consecutive_day
    <span class="hljs-keyword">FROM</span> (
        <span class="hljs-keyword">SELECT</span> <span class="hljs-keyword">SUM</span>(rise_mark) <span class="hljs-keyword">OVER</span> (<span class="hljs-keyword">ORDER</span> <span class="hljs-keyword">BY</span> trade_date) <span class="hljs-keyword">AS</span> days_no_gain
        <span class="hljs-keyword">FROM</span> (
            <span class="hljs-keyword">SELECT</span> trade_date,
                   <span class="hljs-keyword">CASE</span> <span class="hljs-keyword">WHEN</span> closing_price &gt; LAG(closing_price) <span class="hljs-keyword">OVER</span> (<span class="hljs-keyword">ORDER</span> <span class="hljs-keyword">BY</span> trade_date)
                        <span class="hljs-keyword">THEN</span> <span class="hljs-number">0</span> <span class="hljs-keyword">ELSE</span> <span class="hljs-number">1</span> <span class="hljs-keyword">END</span> <span class="hljs-keyword">AS</span> rise_mark
            <span class="hljs-keyword">FROM</span> stock_price
        )
    )
    <span class="hljs-keyword">GROUP</span> <span class="hljs-keyword">BY</span> days_no_gain
);
</code></pre>
<ul>
<li><strong>SQL can be super slow sometimes</strong> 💩</li>
</ul>
<p>If you have ever worked in a real-world situation, you likely had to perform queries on a database containing thousands or even millions of entries.</p>
<p>With such a vast amount of data, SQL often performs poorly in such situations.</p>
<p>Let's take an example of this below SQL statement 👇</p>
<pre><code class="lang-SQL"><span class="hljs-keyword">SELECT</span> TOP <span class="hljs-number">10</span> COLUMN_NAME <span class="hljs-keyword">FROM</span> TABLE_NAME <span class="hljs-keyword">ORDER</span> <span class="hljs-keyword">BY</span> COLUMN_NAME <span class="hljs-keyword">ASC</span>;
</code></pre>
<p>This code in itself is not complex but SQL has to perform the following operations to return the final data.</p>
<ul>
<li>First, sort the entire data.</li>
</ul>
<p><br /></p>
<ul>
<li>From the entire sorted dataset, select the top 10.</li>
</ul>
<blockquote>
<p>Rewriting the previous SQL query in SPL will provide a clear idea of ease when using SPL. To calculate the maximum consecutive days that a stock keeps rising 👇</p>
</blockquote>
<pre><code class="lang-SQL">stock_price.sort(trade_date).group@i(closing_price&lt;closing_price[-1]).max(~.len())
</code></pre>
<p>Although the logic is the same, it is much easier to read/write and is a lot more intuitive. 😄</p>
<hr />
<h2 id="heading-finally-what-is-esproc-spl"><strong>Finally, what is esProc SPL?</strong> 🤩</h2>
<blockquote>
<p>💡 esProc is a scripting language for data processing with rich library functions and powerful syntax.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642343943/f7592aa0-19bf-485a-b9b7-72f71fb4eada.jpeg" alt="esProc SPL" /></p>
<blockquote>
<p>So now you have a basic understanding of SPL, let's discuss what esProc actually is.</p>
</blockquote>
<p>Now that you're aware that SPL is an alternative to SQL, think of esProc as an even superior alternative to traditional SPL with added functionalities. 😲</p>
<p>esProc is a <strong>pure Java</strong> written that makes it easy to add powerful data processing capabilities to your Java 🍵 applications but non-Java applications can invoke esProc through RESTful APIs 🎯.</p>
<p>It provides structured data types and functions 👷 that let you perform operations like filtering, grouping, joining 🪢, and many other computations - similar to what you can do with SQL - but using simple Java code. ✨ esProc implements the basic class libraries entirely within itself.</p>
<blockquote>
<p>🤔 <strong>What platforms can esProc run?</strong></p>
</blockquote>
<p>As it is built purely in Java, it can run smoothly in any OS equipped with <strong>JVM</strong> (Java Virtual Machine), cloud servers, or even containers. 😎</p>
<blockquote>
<p>✨ In short, esProc is blazing fast, independent, and provides advanced scripting capabilities for data handling and analysis over traditional SPL.</p>
</blockquote>
<hr />
<h2 id="heading-lets-wrap-up"><strong>Let's Wrap Up!</strong> ✨</h2>
<p>By now, you have a generic understanding of what <strong>SPL</strong> is. What <strong>esProc</strong> is and how can it come in handy in situations, and whether or not you should <strong>replace SQL</strong> with <strong>SPL</strong> or <strong>esProc</strong>.</p>
<p>After reading this article, if you feel that <strong>esProc</strong> is a great project with huge potential, don't forget to give it a star!</p>
<blockquote>
<p>Please let me what you think of <strong>esProc</strong> in the comments section below. 👇</p>
</blockquote>
<p>So, that is it for this article. Thank you so much for reading! 🎉🫡</p>
]]></content:encoded></item><item><title><![CDATA[✨Be a 10X Linux User with these tools 😎💫]]></title><description><![CDATA[This article lists six great tools for developers to install on their Linux machines. 🎉 Feel free to explore these tools, and star these repositories…]]></description><link>https://shricodev.hashnode.dev/be-a-10x-linux-user-with-these-tools</link><guid isPermaLink="true">https://shricodev.hashnode.dev/be-a-10x-linux-user-with-these-tools</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Wed, 29 Nov 2023 12:25:09 GMT</pubDate><enclosure url="https://media.dev.to/cdn-cgi/image/width=1000,height=420,fit=cover,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvilz7pmj5dfymhhidbc6.gif" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-tldr">TL;DR</h2>
<p>This article lists six great tools for developers to install on their Linux machines. 🎉</p>
<p>Feel free to explore these tools, and star these repositories. Opensource projects need your help! 🙏🏻</p>
<p>Are you ready to become a <strong>10X Linux user</strong>? 🧠 Just kidding, you will not be, 😆 but these tools can definitely speed up your daily workloads. 🏃🏻‍♂️💨</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642511708/8f9c2a85-1bc0-40f5-8131-848e4b679480.gif" alt="Laughing GIF" /></p>
<hr />
<p><strong>1.</strong> <a target="_blank" href="https://ngrok.com/"><strong>Ngrok</strong></a></p>
<blockquote>
<p>💡 Securely expose local servers online.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642513489/74c6d05b-0a99-44de-9615-2a164ecf42c2.jpeg" alt="ngrok cli tool" /></p>
<p>Ngrok helps create a public URL 🌐 for your local web server so you can share your work-in-progress with teammates, clients, or the world. 🌍</p>
<p>Just by running a simple command, Ngrok spins up a secure tunnel from a public endpoint to your computer. This lets you test websites, apps, and APIs on your <code>localhost</code> without having to deploy anything.</p>
<p>Ngrok supports HTTP and TCP traffic so you can tunnel almost anything 🤯 - web servers, or even databases indirectly. It also offers useful features like custom subdomains and replaying/inspecting traffic.</p>
<p>Ngrok helps in these major areas:</p>
<ul>
<li>Temporarily share a website that is only running on your development machine 🖥️.</li>
</ul>
<ul>
<li>Develop any services that consume webhooks 🪝.</li>
</ul>
<ul>
<li>Debug 🧑‍💻 a web service by inspecting web traffic.</li>
</ul>
<blockquote>
<p>✨ I don't use this tool very often, but I remember I used this tool for the very first time a few years back to show my friend the website I had created locally.</p>
</blockquote>
<hr />
<p><strong>2.</strong> <a target="_blank" href="https://github.com/junegunn/fzf"><strong>fzf</strong></a></p>
<blockquote>
<p>💡 A command-line fuzzy finder.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642514766/38935b9f-dfbc-4372-9cbb-577deb170ce2.jpeg" alt="fzf cli tool" /></p>
<p>Fzf is a useful tool for anyone who spends a lot of time working in the terminal 🖥️. It acts as a fuzzy finder, allowing you to quickly search 🔍 and filter through data lists like files, command history, git commits, and more.</p>
<p>Here's how it works. When you run an <code>fzf</code> command, it instantly filters the list below as you type to show only matching results.</p>
<p>Here's a possible use case. 😯</p>
<pre><code class="lang-bash">ls -la | fzf
</code></pre>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642516848/974f6b0c-734b-4403-b6f8-dd014cd73363.png" alt="fzf tool use case" /></p>
<p>This interactive approach makes it fast and easy to find what you're looking for, even on long lists. Fuzzy matching means it will find partial matches, so you don't have to type the full name. 🤯</p>
<hr />
<p><strong>3.</strong> <a target="_blank" href="https://github.com/rupa/z"><strong>Z</strong></a></p>
<blockquote>
<p>💡 Jump around directories without knowing the entire path.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642518514/ad998b81-556b-4757-a58d-648b1cc2f259.jpeg" alt="z cli tool" /></p>
<p>Have you ever been working on a project with a deeply nested folder structure and had to constantly type out long directory 📂 paths using the cd command? Z solves that problem. 😉</p>
<p>Once you install and start using Z, it tracks the directories you access most often. Then, instead of typing the full path, you can enter a short name to instantly jump 🦘 to that folder.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642519837/9e3d4f55-5d9c-4093-b2ee-930a9b25d9d4.jpeg" alt="Z tool usage" /></p>
<p>Z uses <strong>two factors</strong> to determine which directories to track for you: how <strong>frequently</strong> you access a folder and how <strong>recently</strong> you accessed it.</p>
<p>So the folders you use the most will get the shortest names.</p>
<hr />
<p><strong>4.</strong> <a target="_blank" href="https://github.com/ranger/ranger"><strong>Ranger</strong></a></p>
<blockquote>
<p>💡 A VIM-inspired file manager for the console.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642521268/b08847dd-04ce-40a7-97b4-5eb214f4fd0d.jpeg" alt="ranger cli file manager" /></p>
<p>Ranger is a <strong>vim-inspired</strong> file manager 👮🏻 with a console interface. It provides an easy-to-use and efficient way to navigate and view the file system.</p>
<p>Some major features of Ranger are:</p>
<ul>
<li>Multi-column ✨ display</li>
</ul>
<ul>
<li>Preview 📂 of the selected file/directory</li>
</ul>
<ul>
<li>Common file operations (<code>create/chmod/copy/delete</code>/...)</li>
</ul>
<ul>
<li>Renaming multiple files at once</li>
</ul>
<ul>
<li>VIM-like console and hotkeys ⌨️</li>
</ul>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642523321/84afad5a-dbce-4d7d-83f6-140b7145c3cb.png" alt="ranger cli file manager" /></p>
<hr />
<p><strong>5.</strong> <a target="_blank" href="https://www.tecmint.com/ncdu-a-ncurses-based-disk-usage-analyzer-and-tracker/"><strong>Ncdu</strong></a></p>
<blockquote>
<p>💡 Review files and the disk space being used on Linux systems.</p>
</blockquote>
<p>Ncdu is a useful command line tool that helps you manage your disk space efficiently.</p>
<p>It shows you a visual representation of how space is used on your disk drives, making it easy to identify large files and folders that are taking up the most space.</p>
<p>When you run Ncdu, it scans 🔍 your file system and displays an interactive list in your terminal, showing you each folder on your disk drives and how much space they are using. You can navigate this list to drill down and see the space usage of subfolders 📂.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642525474/17069d4e-a49f-4ccb-a5a3-61ec17f883b4.png" alt="ncdu cli tool" /></p>
<p>Its simple interface makes it easy to use yet powerful for analyzing your disk usage and optimizing your storage.</p>
<p>If you struggle with your disk space 💾 filling up, Ncdu can be a handy tool to identify the culprits and take action. 👮🏻</p>
<hr />
<p><strong>6.</strong> <a target="_blank" href="https://github.com/ogham/exa"><strong>Exa</strong></a> - <em>Good to have</em></p>
<blockquote>
<p>💡 Colorful replacement of <code>ls</code> command.</p>
</blockquote>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642528820/ae916053-0622-4368-bc51-82626dff6695.jpeg" alt="exa cli tool" /></p>
<blockquote>
<p>🚨 Exa is now unmaintained, use <a target="_blank" href="https://github.com/eza-community/eza">eza</a>. It is a fork of exa with a couple of features added. But I am still using Exa and it works pretty fine. They are pretty much the same.</p>
</blockquote>
<p>Exa is a modern replacement for the old <code>ls</code> command that you use in the terminal to list files and folders. It gives you a more colorful 🎨, detailed, and easy-to-read listing of the files in your current directory.</p>
<p>When you run <strong>exa</strong> instead of <code>ls</code>, you'll see a few nice improvements. First, it color codes the file types using different colors and icons (enable or disable) 💬 making it easier to visually distinguish files at a glance.</p>
<p>It also shows you extra information like file permissions, owners, sizes, and more.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642530535/85b85bd4-830a-4a1e-ba4d-b784d28e12b2.png" alt="exa cli tool" /></p>
<hr />
<blockquote>
<p>If you think of any other handy tools that I haven't covered in this article, please share them in the comments section below. 👇🏻</p>
</blockquote>
<p>So, that is it for this article. Thank you so much for reading! 🎉🫡</p>
]]></content:encoded></item><item><title><![CDATA[Drizzle: ORM with SQL Superpowers🚀]]></title><description><![CDATA[💡 Hey folks, if you've clicked on this article, I assume you're familiar with Prisma or have worked with SQL before. let's discuss Drizzle⚡ - the hot new…]]></description><link>https://shricodev.hashnode.dev/drizzle-orm-with-sql-superpowers</link><guid isPermaLink="true">https://shricodev.hashnode.dev/drizzle-orm-with-sql-superpowers</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Mon, 25 Sep 2023 13:35:38 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642318820/4404266d-f769-4898-9a0d-ef057cb03ef7.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<blockquote>
<p>💡 Hey folks, if you've clicked on this article, I assume you're familiar with Prisma or have worked with SQL before.</p>
</blockquote>
<p>let's discuss Drizzle⚡ - the hot new open-source alternative to Prisma!</p>
<hr />
<h2 id="heading-what-is-drizzle">🤷‍♂️ What is Drizzle?</h2>
<p>Drizzle is a TypeScript ORM for SQL databases(<code>PostgreSQL</code>, <code>MySQL</code>, ...) designed with maximum type safety.</p>
<p>It comes with a handy built-in drizzle-kit CLI which helps in the SQL migrations if we make any changes to the scheme file.</p>
<p>➡️ Installation - <em>NodeJS-PostgreSQL</em>:</p>
<pre><code class="lang-bash">pnpm add drizzle-orm pg
pnpm add -D drizzle-kit @types/pg
</code></pre>
<h3 id="heading-how-do-you-choose-between-prisma-and-drizzle">How do you choose between Prisma and Drizzle?</h3>
<blockquote>
<p>👉 If you are familiar with SQL queries go with Drizzle, else Prisma is a bit intuitive for beginners. It's this simple! 😉</p>
</blockquote>
<hr />
<h2 id="heading-schema-and-migrations">🧑‍💻 Schema and Migrations</h2>
<p>I will demonstrate how the Schema and Migrations work in Drizzle by building a <em>NodeJS + PostgreSQL (with docker)</em> project.</p>
<p><strong>1.</strong> Initialize a NextJS project</p>
<pre><code class="lang-bash">pnpx create-next-app@latest
</code></pre>
<p><strong>2.</strong> Now, let's initialize the docker-compose.yaml file in the root of the project with the following contents</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642312649/29f96e93-27c7-4a2e-a5aa-ce1379d3aa2b.png" alt="Docker Compose Yaml File" /></p>
<p>We simply host a PostgreSQL service on port <strong>5432</strong> (default) and map its original volume to <strong>./data/db</strong> (for <em>data persistance</em>) in our current folder, with the admin interface on port <strong>8080</strong>.</p>
<p><strong>NOTE:</strong> We are also using environment variables to store credentials for the database.</p>
<p><strong>3.</strong> Spin up the docker container</p>
<p>Go to the root of your project and run this command. It's better to have it up and running since we will be performing migrations and all.</p>
<pre><code class="lang-bash">docker compose -f docker-compose.yaml up
</code></pre>
<blockquote>
<p>💡 If you are on Windows, make sure to have your docker daemon running before performing this step.</p>
</blockquote>
<p><strong>4.</strong> Now, we set up the drizzle config in the drizzle.config.ts file in the root folder of the project. Make sure to declare an env named <code>DATABASE_URL</code> with the connection string in .env.local.</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> <span class="hljs-keyword">type</span> { Config } <span class="hljs-keyword">from</span> <span class="hljs-string">"drizzle-kit"</span>;
<span class="hljs-keyword">import</span> * <span class="hljs-keyword">as</span> dotenv <span class="hljs-keyword">from</span> <span class="hljs-string">"dotenv"</span>;

dotenv.config({ path: <span class="hljs-string">".env.local"</span> });

<span class="hljs-keyword">if</span> (!process.env.DATABASE_URL) <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Error</span>(<span class="hljs-string">"DATABASE_URL is not set"</span>);

<span class="hljs-keyword">export</span> <span class="hljs-keyword">default</span> {
  schema: <span class="hljs-string">"./src/lib/db/*"</span>, <span class="hljs-comment">// any schema file inside the /db folder.</span>
  out: <span class="hljs-string">"./drizzle"</span>,
  driver: <span class="hljs-string">"pg"</span>,
  dbCredentials: {
    connectionString: process.env.DATABASE_URL,
  },
} satisfies Config;
</code></pre>
<p><strong>5.</strong> Create table schema. I am taking the schema file from my previous project.</p>
<p>Declare a <code>schema.ts</code> in <strong>/src/lib/db/schema.ts</strong> as shown in the drizzle config file.</p>
<pre><code class="lang-plaintext">project-name
└── src/
    ├── app/
    │   └── ...
    ├── lib/
    │   └── db/
    │       ├── schema.ts
    │       └── ...
    └── ...
</code></pre>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> {
  integer,
  pgEnum,
  pgTable,
  serial,
  text,
  timestamp,
  varchar,
} <span class="hljs-keyword">from</span> <span class="hljs-string">"drizzle-orm/pg-core"</span>;

<span class="hljs-comment">// role can be either "user" or "bot"</span>
<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> userBotEnum = pgEnum(<span class="hljs-string">"user_bot_enum"</span>, [<span class="hljs-string">"user"</span>, <span class="hljs-string">"bot"</span>]);

<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> chats = pgTable(<span class="hljs-string">"chats"</span>, {
  id: serial(<span class="hljs-string">"id"</span>).primaryKey(),
  docName: text(<span class="hljs-string">"doc_name"</span>).notNull(),
  docUrl: text(<span class="hljs-string">"doc_url"</span>).notNull(),
  createdAt: timestamp(<span class="hljs-string">"created_at"</span>).notNull().defaultNow(),
  userId: varchar(<span class="hljs-string">"user_id"</span>, { length: <span class="hljs-number">255</span> }).notNull(),
  <span class="hljs-comment">// This is specific to the AWS S3</span>
  fileKey: text(<span class="hljs-string">"file_key"</span>).notNull(),
});

<span class="hljs-comment">// export the type declaration of chats.</span>
<span class="hljs-keyword">export</span> <span class="hljs-keyword">type</span> ChatType = <span class="hljs-keyword">typeof</span> chats.$inferSelect;

<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> messages = pgTable(<span class="hljs-string">"messages"</span>, {
  id: serial(<span class="hljs-string">"id"</span>).primaryKey(),
  chatId: integer(<span class="hljs-string">"chat_id"</span>)
    .references(<span class="hljs-function">() =&gt;</span> chats.id)
    .notNull(),
  content: text(<span class="hljs-string">"content"</span>).notNull(),
  createdAt: timestamp(<span class="hljs-string">"created_at"</span>).notNull().defaultNow(),
  role: userBotEnum(<span class="hljs-string">"role"</span>).notNull(),
});
</code></pre>
<p>The schema file should be pretty intuitive. We are simply creating a schema for two tables <code>chats</code> and <code>messages</code>. The only thing new is the <code>references()</code> function.</p>
<p>Basically, we are creating a relation between the <code>chatId</code> in the messages and <code>chats.id</code> in the chats which helps us get the chatId specific to a chat in the <code>messages</code> table.</p>
<p><strong>6.</strong> After writing the schema file. we also need a way to push the migrations to the database itself.</p>
<p>Create a migrate.ts in <strong>/src/lib/db</strong>.</p>
<pre><code class="lang-typescript"><span class="hljs-keyword">import</span> { Pool } <span class="hljs-keyword">from</span> <span class="hljs-string">"pg"</span>;
<span class="hljs-keyword">import</span> { migrate } <span class="hljs-keyword">from</span> <span class="hljs-string">"drizzle-orm/node-postgres/migrator"</span>;
<span class="hljs-keyword">import</span> { drizzle } <span class="hljs-keyword">from</span> <span class="hljs-string">"drizzle-orm/node-postgres"</span>;

<span class="hljs-keyword">import</span> * <span class="hljs-keyword">as</span> dotenv <span class="hljs-keyword">from</span> <span class="hljs-string">"dotenv"</span>;

dotenv.config({ path: <span class="hljs-string">".env.local"</span> });

<span class="hljs-keyword">if</span> (!process.env.DATABASE_URL)
  <span class="hljs-keyword">throw</span> <span class="hljs-keyword">new</span> <span class="hljs-built_in">Error</span>(<span class="hljs-string">"Cannot migrate. DATABASE_URL is not set"</span>);

<span class="hljs-keyword">const</span> pool = <span class="hljs-keyword">new</span> Pool({
  connectionString: process.env.DATABASE_URL,
});
<span class="hljs-keyword">export</span> <span class="hljs-keyword">const</span> db = drizzle(pool);

<span class="hljs-keyword">async</span> <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">main</span>(<span class="hljs-params"></span>) </span>{
  <span class="hljs-built_in">console</span>.log(<span class="hljs-string">"Running your migrations..."</span>);
  <span class="hljs-keyword">await</span> migrate(db, { migrationsFolder: <span class="hljs-string">"drizzle"</span> });
  <span class="hljs-built_in">console</span>.log(<span class="hljs-string">"Woohoo! Migrations completed!"</span>);
  <span class="hljs-keyword">return</span>;
}

main().catch(<span class="hljs-function">(<span class="hljs-params">err</span>) =&gt;</span> {
  <span class="hljs-built_in">console</span>.error(err);
  process.exit(<span class="hljs-number">1</span>);
});
</code></pre>
<p>We are connecting to the Postgres server with the <code>pg</code> driver using the Pool connection. Now the migration script is ready.</p>
<p>We are now finally ready to push our schemas to the database. Add the following in the <code>scripts</code> section in the package.json</p>
<pre><code class="lang-json">    <span class="hljs-string">"generate"</span>: <span class="hljs-string">"drizzle-kit generate:pg"</span>,
    <span class="hljs-string">"push"</span>: <span class="hljs-string">"node -r esbuild-register src/lib/db/migrate.ts"</span>,
    <span class="hljs-string">"studio"</span>: <span class="hljs-string">"drizzle-kit studio --host 127.0.0.1"</span>
</code></pre>
<ul>
<li><strong>generate</strong> - This is the command responsible for generating the migration files. Upon running this command you should see a new drizzle folder created on the topmost level of the project with the <code>.sql</code> files. <code>generate:pg</code> is using the Postgres (pg) driver to generate the migrations.</li>
</ul>
<ul>
<li><strong>push</strong> - This is the command responsible for pushing the migration files to the actual docker PostgreSQL database hosted on our local machine. We are actually executing the migrate.ts file that we wrote earlier.</li>
</ul>
<ul>
<li><strong>studio</strong> - This is not so necessary command, but this is something you can use as an alternative to the <strong>adminer</strong> interface. Upon running this script, you get a web interface to interact with the database in the port <em>4983</em> - localhost:4983</li>
</ul>
<p>Finally, run these commands sequentially. And now you should have your drizzle studio running as well. Open localhost:4983 or localhost:8080 for <em>adminer</em> interface.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642314980/9a92356c-1fbb-43e5-a227-6c3b2e632037.gif" alt="Demo GIF" /></p>
<p>See, it is this simple. Now, you can prepare a schema file, generate its migrations, and push the changes to the database.🫡</p>
<hr />
<h2 id="heading-comparison-with-other-orms">⚖️ Comparison with other ORMs</h2>
<p>Drizzle is designed to be both simple and customizable while still staying fast.</p>
<p>When comparing Drizzle to other ORMs, such as Prisma, it becomes clear that there are differences in their approach. Drizzle focuses on providing a <em>flexible query builder, resembling SQL usage</em>, and <strong>excels in serverless</strong>🚀 environments as it's built on top of TypeScript.</p>
<p>On the other hand, <strong>TypeORM</strong>, a popular ORM, <em>emphasizes type annotations and schema management</em>. Drizzle, while still lightweight, offers a compelling alternative for query building and database access.</p>
<h3 id="heading-is-drizzle-suitable-for-your-production-app">😕 Is Drizzle suitable for your production app?</h3>
<p>Drizzle is a fairly new ORM, so there may still be some work to be done. However, it is already compatible with most of the SQL databases out there.</p>
<p>If you are familiar with relational databases, then Drizzle can be a go-to choice for any production-level application. 💯</p>
<hr />
<h2 id="heading-final-thoughts">🚀 Final Thoughts!</h2>
<blockquote>
<p>✨ As they say — When it comes to Drizzle - we're a thin TypeScript layer on top of SQL. The ORM's main philosophy is "<strong>If you know SQL, you know Drizzle ORM</strong>"</p>
</blockquote>
<p><mark>They aren't trying to be the next Prisma. It's more of a query builder than a full-fledged ORM, like </mark> <strong><mark>Kysely</mark></strong> <mark> or </mark> <strong><mark>Knex</mark></strong><mark>.</mark></p>
<p>It is performance-centric, has fully-fledged joints, and lets developers fall in 😍 with SQL again!</p>
<p>Lately, they have gained a lot of popularity over their repository.📈</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642317367/20d6943f-ee82-43d2-916b-5e7a9e4f87ac.png" alt="Star Increase Chart in Drizzle Repository" /></p>
<p>So, that is it for this article. I hope this is useful for every one of you. 🤗</p>
<p>See you next time✌️</p>
<p><img src="https://media.giphy.com/media/kGCuRgmbnO9EI/giphy.gif" alt="bye gif" /></p>
<blockquote>
<p>Drop your thoughts in the comments section below. 👇</p>
</blockquote>
<hr />
<p>Feel free to follow me to get notified when new articles are out 😉</p>
<p>Let's stay connected.</p>
<ul>
<li><p>GitHub: https://github.com/shricodev</p>
</li>
<li><p>LinkedIn: https://linkedin.com/in/shrijal007</p>
</li>
<li><p>X (Twitter): https://twitter.com/shricodevv</p>
</li>
</ul>
]]></content:encoded></item><item><title><![CDATA[Git Bisect - The Underdog Git Command🔥]]></title><description><![CDATA[Do you know which Git command often flies under the radar, despite its incredible usefulness? Enter Git Bisect - the unsung hero of git. 🚀 In this article…]]></description><link>https://shricodev.hashnode.dev/git-bisect-the-underdog-git-command</link><guid isPermaLink="true">https://shricodev.hashnode.dev/git-bisect-the-underdog-git-command</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Tue, 15 Aug 2023 12:45:00 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642283677/3d9f7d56-0bd7-43ee-af8d-25fe31ad17f3.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Do you know which Git command often flies under the radar, despite its incredible usefulness? Enter Git Bisect - the unsung hero of git. 🚀</p>
<p>In this article we will cover this underdog tool, revealing how it simplifies hunting down bugs. ✨</p>
<hr />
<h2 id="heading-when-to-use-git-bisect">When to use Git Bisect? 🤔</h2>
<p>Git Bisect is like a detective for your code! When your project has a bug and you're not sure which change caused it, Git Bisect helps you find the sneaky culprit by narrowing down the bad commit/code step by step. 😲</p>
<p>Super useful when you want to track down that sneaky bug hiding in your project! 🐛</p>
<blockquote>
<p>🔴 But let me make it clear, <code>git bisect</code> is not a git command you will use very often. Nonetheless, it's one of the most helpful and impressive Git commands.</p>
</blockquote>
<hr />
<h2 id="heading-basic-git-bisect-workflow">Basic <code>git bisect</code> workflow 🎯</h2>
<blockquote>
<p>💡What <code>git bisect</code> does is, it divides the revision commits into a good part and a bad part by testing specific commits chosen by a binary search. 
We will not go through every other nitty-gritty detail in this article. For more information visit <a target="_blank" href="https://git-scm.com/docs/git-bisect">here</a>.</p>
</blockquote>
<ol>
<li><p>You start by saying, "Hey Git, let's find that bug!" - <code>git bisect start</code></p>
</li>
<li><p>Git checks out all the commits where the bug is there and where it's not. - <code>git bisect good/bad</code></p>
</li>
<li><p>You check and tell Git if the bug is still there. - <code>Continue...</code></p>
</li>
<li><p>Git keeps showing you commits until you catch the sneaky bug. Bug busted! 🐛🎉 - <code>git bisect reset</code></p>
</li>
</ol>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642278713/90c20436-92bf-453e-99de-085e0ee41ec7.png" alt="Git Bisect Workflow" /></p>
<hr />
<h2 id="heading-hands-on">Hands On! ⚒️</h2>
<p>Let's implement our little knowledge of what we just learned about <code>git bisect</code> and its basic workflow.</p>
<h3 id="heading-scenario">Scenario 🎥</h3>
<p>My blog site was running smoothly until a few days back, but then something went wrong with a recent update. 🥲</p>
<p>But here we can use the magic of the git bisect to find out which commit caused the issue.</p>
<p>First, let’s run the <code>git log --oneline</code> command to see the commit history:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642280255/9613cce8-bb3d-4b1e-8dbf-4f779ba1cb01.png" alt="Git Log Output" /></p>
<blockquote>
<p>Here the bug introducing commit is <code>690a8a0</code>, but let's pretend we don't know which commit introduced the error. 🤡</p>
</blockquote>
<p>For the first good commit, we will pick <code>5e2bae3</code>. As far as we know, that’s the last commit that doesn’t have the error.</p>
<p>For the first bad commit, we can choose the latest commit <code>464c3c3</code> since the app is not currently working. ☹️</p>
<p>Now, let's start <code>git bisect</code> and provide the initial good and bad commits.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642281862/51167553-143e-4e0b-a9a6-3efa4fc969ec.png" alt="Git Bisect" /></p>
<p>Now, git checks out every other commit in the commit revision. Now, we need to test our app manually on every checkout. The commit we're currently checked out <code>cdd57cc</code> seem to not have the issue. So, we mark it good. ✅</p>
<p>Now, we are only left with these two commits - <code>690a8a0</code> <code>c2a0519</code></p>
<p>Git now checks out the buggy commit <code>690a8a0</code> which on testing manually seem to have caused the issue. Finally, we have found that buggy🐛 commit.</p>
<p>Once you’ve finished your testing, you can reset bisect using
<code>git bisect reset</code>.</p>
<p>That is it for this article! 😎</p>
<p>Here, we covered the manual <code>git bisect</code>, but if you have a test script that you can run to validate commits, you can use it to automate the whole thing with the command <code>git bisect run &lt;script&gt;</code></p>
<hr />
<p>Feel free to follow me to get notified when new articles are out 😉</p>
<p>Let's stay connected👋</p>
<ul>
<li>GitHub: https://github.com/shricodev</li>
<li>LinkedIn: https://linkedin.com/in/shrijal007</li>
</ul>
<p>See you next time✌️</p>
]]></content:encoded></item><item><title><![CDATA[5 Best Practices when deploying a Docker Container🚀]]></title><description><![CDATA[Ahoy, Docker adventurers!🚀 Ready to set sail into the Docker universe? Get ready for simple yet smart container advice – let's make Docker a breeze! 🐳…]]></description><link>https://shricodev.hashnode.dev/5-best-practices-when-deploying-a-docker-container</link><guid isPermaLink="true">https://shricodev.hashnode.dev/5-best-practices-when-deploying-a-docker-container</guid><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Tue, 08 Aug 2023 08:08:32 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642492833/3443984b-4333-4e58-94ab-1283d1d80d78.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Ahoy, Docker adventurers!🚀 Ready to set sail into the Docker universe? Get ready for simple yet smart container advice – let's make Docker a breeze! 🐳</p>
<hr />
<h2 id="heading-best-practices">Best Practices🚩</h2>
<p>Let's get straight into it. These are my top 5 best practices to follow when creating a docker container.  </p>
<hr />
<h3 id="heading-use-smaller-base-image">✅ Use Smaller Base Image</h3>
<blockquote>
<p>💡If possible, never ever use big base images.</p>
</blockquote>
<p>In Docker Hub when choosing for any images, you will see a bunch of image with it's OS.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1723642491213/eb411035-699c-477a-a980-6ca7a2282b07.png" alt="Best Practice 1 - Docker Image with OS" /></p>
<p>If there is an <code>alpine OS</code> version of the image you are looking for then try to use it always.</p>
<p>It comes with just the tools you need and is very lightweight compared to any other versions🔥.</p>
<p>By doing so you need a very <strong>small storage capacity</strong> for the image itself and also reduce the <strong>attack surface</strong> since it does not come with whole bunch of unnecessary tools.</p>
<p>There is a clear performance difference between choosing a <strong>lightweight alpine image vs a full blown debian.</strong></p>
<p><strong>Debian: 🐢</strong></p>
<pre><code class="lang-bash">time docker run --rm debian sh -c <span class="hljs-string">"apt-get update &amp;&amp; apt-get install curl"</span>

real    0m27.928s
user    0m0.019s
sys    0m0.077s
</code></pre>
<p><strong>Alpine:⚡</strong></p>
<pre><code class="lang-bash">time docker run --rm alpine sh -c <span class="hljs-string">"apk update &amp;&amp; apk add curl"</span>

real    0m5.698s
user    0m0.008s
sys    0m0.037s
</code></pre>
<hr />
<h3 id="heading-always-use-dockerignore-file">✅ Always use <code>.dockerignore</code> file</h3>
<blockquote>
<p>💡Use <code>.dockerignore</code> to ignore files that are not needed for your container.</p>
</blockquote>
<p>Don't push the unnecessary build folders such as <code>dist</code> or <code>build</code> and files such as <code>README.md</code> and <code>Dockerfile</code> itself. They are not needed to run our container and take up unnecessary space.</p>
<p>So, with <code>.dockerignore</code> we reduce the image size.😉</p>
<hr />
<h3 id="heading-use-specific-docker-image-version">✅ Use Specific Docker Image Version</h3>
<blockquote>
<p>💡When choosing a base image never choose the latest tag.
Use a specific version.</p>
</blockquote>
<p>This is a very bad practice ❌</p>
<pre><code class="lang-dockerfile"><span class="hljs-keyword">FROM</span> python
</code></pre>
<p>You know, in the future, even the smallest change could accidentally bring in a <strong>bug</strong>, or worse, <strong>break your application</strong>. 🥲</p>
<p>This is a good practice ✅</p>
<pre><code class="lang-dockerfile"><span class="hljs-keyword">FROM</span> python:<span class="hljs-number">3.11</span>
</code></pre>
<hr />
<h3 id="heading-do-not-use-the-root-user">✅ Do not use the <code>root</code> user</h3>
<blockquote>
<p>💡Try to use the least privileged user when running the container.</p>
</blockquote>
<p>By default, when <code>Dockerfile</code> does not specify a user, it uses the <strong>root</strong> user. You do not want to run the container with root privilege. ❌</p>
<p>Running the container with root access makes it easier for a hacker to <strong>exploit the application</strong> inside the container.</p>
<p>To avoid this create a specific user and use it to run all the processes.</p>
<pre><code class="lang-dockerfile">
<span class="hljs-comment"># Setup group and user</span>
<span class="hljs-keyword">RUN</span><span class="bash"> groupadd -r limiteduser &amp;&amp; useradd -g limiteduser limiteduser</span>
<span class="hljs-keyword">RUN</span><span class="bash"> chown -R limiteduser:limiteduser /app</span>

<span class="hljs-comment"># This switches the user from root to limiteduser</span>
<span class="hljs-keyword">USER</span> limiteduser
</code></pre>
<hr />
<h3 id="heading-use-multi-stage-builds">✅ Use multi-stage builds</h3>
<blockquote>
<p>💡Again, this helps reduce the image size.</p>
</blockquote>
<p>Imagine you're baking a cake: you gather ingredients, mix them, and bake. But what if you could magically remove all the mixing bowls and just keep the final cake?😲</p>
<p>That's what multi-stage builds do! They help build containers in stages, and you only keep the necessary parts.</p>
<p>It allows you to use multiple temporary image during build, but keep only the final image.</p>
<p>This is a sample <code>Dockerfile</code> example that uses multi-stage build. </p>
<pre><code class="lang-dockerfile"><span class="hljs-comment"># Stage 1: Build the application</span>
<span class="hljs-keyword">FROM</span> python:<span class="hljs-number">3.8</span> AS build
<span class="hljs-keyword">WORKDIR</span><span class="bash"> /app</span>
<span class="hljs-keyword">COPY</span><span class="bash"> requirements.txt .</span>
<span class="hljs-keyword">RUN</span><span class="bash"> pip install --no-cache-dir -r requirements.txt</span>
<span class="hljs-keyword">COPY</span><span class="bash"> app.py .</span>

<span class="hljs-comment"># Stage 2: Create the final container</span>
<span class="hljs-keyword">FROM</span> python:<span class="hljs-number">3.8</span>-slim
<span class="hljs-keyword">WORKDIR</span><span class="bash"> /app</span>
<span class="hljs-keyword">COPY</span><span class="bash"> --from=build /app /app</span>
<span class="hljs-keyword">CMD</span><span class="bash"> [<span class="hljs-string">"python"</span>, <span class="hljs-string">"app.py"</span>]</span>
</code></pre>
<hr />
<p>So, that is it for this article. Hope this is useful for every one of you. 🤗</p>
<blockquote>
<p>If you think of any other helpful tips that I haven't covered in this article, please don't hesitate to share them in the comments section below. 🔥</p>
</blockquote>
<hr />
<p>Let's stay connected👋</p>
<ul>
<li>GitHub: https://github.com/shricodev</li>
<li>LinkedIn: https://linkedin.com/in/shrijal007</li>
</ul>
<p>See you next time✌️</p>
]]></content:encoded></item><item><title><![CDATA[Troubleshoot your Node.js code like a Pro!]]></title><description><![CDATA[I'll walk you through real examples, making the process super easy to grasp. No more frustrations – let's dive in and become Node.js troubleshooting…]]></description><link>https://shricodev.hashnode.dev/troubleshoot-your-nodejs-code-like-a-pro</link><guid isPermaLink="true">https://shricodev.hashnode.dev/troubleshoot-your-nodejs-code-like-a-pro</guid><category><![CDATA[Node.js]]></category><category><![CDATA[debugging]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Express]]></category><category><![CDATA[WeMakeDevs]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Wed, 26 Jul 2023 03:10:11 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1690211581017/e73a9086-18ce-4b4f-a5a6-d3bd98f96326.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Ever felt stuck while trying to troubleshoot Node.js issues and found that the internet has disappointingly limited content to help you out? In this article, I've got your back with some awesome techniques to troubleshoot Node.js like a champ.</p>
<p>I'll walk you through real examples, making the process super easy to grasp. No more frustrations – let's dive in and become Node.js troubleshooting experts together! 🚀</p>
<h2 id="heading-tools-and-techniques">Tools and Techniques</h2>
<ol>
<li><strong>Using Chrome Debugger:</strong></li>
</ol>
<div data-node-type="callout">
<div data-node-type="callout-emoji">💡</div>
<div data-node-type="callout-text">There is an even better way than this, just stay with me for some time</div>
</div>

<p>Since Chrome by default does not support Node.js debugging. So for this to work you need to tweak the start script in <code>package.json.</code></p>
<pre><code class="lang-json">  <span class="hljs-string">"scripts"</span>: {
    <span class="hljs-attr">"start"</span>: <span class="hljs-string">"node --inspect app.js"</span>
  },
</code></pre>
<p>Note the <mark>--inspect</mark> flag that I am using, make sure to add it.</p>
<p>Let's take this sample code:</p>
<pre><code class="lang-javascript"><span class="hljs-keyword">import</span> express <span class="hljs-keyword">from</span> <span class="hljs-string">"express"</span>
<span class="hljs-keyword">const</span> app = express();

app.get(<span class="hljs-string">'/'</span>, <span class="hljs-function">(<span class="hljs-params">req, res</span>) =&gt;</span> {
    <span class="hljs-built_in">console</span>.log(<span class="hljs-string">"I am inside the server!"</span>);
    <span class="hljs-keyword">debugger</span>;
    res.send(<span class="hljs-string">"Response"</span>);
})

app.listen(<span class="hljs-number">4000</span>);
<span class="hljs-built_in">console</span>.log(<span class="hljs-string">"Server Started"</span>);
</code></pre>
<p>The <mark>debugger</mark> keyword acts like a breakpoint. Now, whenever you start up a server with <code>yarn run start</code>, you will notice that it indicates a debugger is running. Next, open up Chrome DevTools and wait for a few seconds. You will see a Node.js icon appearing at the top of the DevTools panel. Click on this icon, and now in the Sources tab, you will notice that the debugger breakpoint has been hit. Now you can debug your Node.js code just like you would debug regular JavaScript code.</p>
<p>If you are still not convinced about this approach and want something even better, you can utilize it directly within your Code Editor. The next approach is something you would love and what I use as well.</p>
<ol>
<li><strong>Using Built-In Editor debugger:</strong></li>
</ol>
<p>In this section, I will be using VScode and the approach should be pretty similar in any other code editor.</p>
<p>Firstly, we need to set up the debugger configuration. We can do that by creating a <code>launch.json</code> file.</p>
<p>Go to the <strong><em>Run and Debug</em></strong> section in the left pane of the sidebar. Click on <code>"Create a launch.json"</code> and select Node.js.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1690274759544/57299375-a6b8-4a35-91a3-d65108d8d25e.png" alt class="image--center mx-auto" /></p>
<p>This will create a <code>launch.json</code> file in the root of the project inside the <code>.vscode</code> folder. The default configuration should be similar to the below json.</p>
<pre><code class="lang-json">{
  <span class="hljs-attr">"version"</span>: <span class="hljs-string">"0.2.0"</span>,
  <span class="hljs-attr">"configurations"</span>: [
    {
      <span class="hljs-attr">"type"</span>: <span class="hljs-string">"node"</span>,
      <span class="hljs-attr">"request"</span>: <span class="hljs-string">"launch"</span>,
      <span class="hljs-attr">"name"</span>: <span class="hljs-string">"Launch Program"</span>,
      <span class="hljs-attr">"skipFiles"</span>: [
        <span class="hljs-string">"&lt;node_internals&gt;/**"</span>
      ],
      <span class="hljs-attr">"program"</span>: <span class="hljs-string">"${workspaceFolder}\\jobs-api\\mywork\\app.js"</span>
    }
  ]
}
</code></pre>
<p>Since we do not want to launch our app in debug mode rather we want to attach our debugger to an already running process. For more information on what is the difference between <code>Launch</code> and <code>Debug</code> visit <a target="_blank" href="https://code.visualstudio.com/docs/editor/debugging#_launch-versus-attach-configurations">here</a>.</p>
<p>On the bottom right corner of the launch.json file click on <mark>Add Configuration</mark> and select <code>Node.js: Attach to Process</code> and also select the same in the debugger dropdown.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1690278250044/85b638ac-450a-4900-8ec0-af268e6803f9.png" alt class="image--center mx-auto" /></p>
<p>And now the process is pretty straightforward. Simply add a breakpoint wherever you want in the <code>app.js</code> file or any controllers you have set up inside the app.</p>
<p>Spin up the server as usual and make a <mark>GET/POST</mark> request to that particular route. You can do this either in the command line itself with <code>curl</code> command or by navigating to the particular URL in the browser.</p>
<p>Now the breakpoint should trigger in your VScode.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1690286755245/5d15e5bf-8d9d-4c5e-860b-f026b68a78b8.gif" alt class="image--center mx-auto" /></p>
<p>As you can clearly see, the curl command is not returning anything because the breakpoint is triggered. So now we can use the VScode <code>step in, pause, step over</code> tools. If you are having trouble refer to this GIF below:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1690287700551/871c9730-ea73-4fcd-af1f-225f71122f61.gif" alt class="image--center mx-auto" /></p>
<p>So now you must have a basic understanding of how to debug Node.js code directly in the browser and in the Code Editor as well. Congratulations!🎊🎊</p>
<h2 id="heading-implementation-in-real-world-code">Implementation in Real World Code</h2>
<p>Enough of the basic understanding, now we will practice it in a real-life code where we use all these things we have learned and troubleshoot the reason for the failing code.</p>
<p>Let's take this failing code:</p>
<pre><code class="lang-javascript"><span class="hljs-keyword">import</span> express <span class="hljs-keyword">from</span> <span class="hljs-string">"express"</span>;
<span class="hljs-keyword">const</span> app = express();

<span class="hljs-keyword">const</span> port = <span class="hljs-number">3000</span>;

<span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">calculateFactorial</span>(<span class="hljs-params">n</span>) </span>{
  <span class="hljs-keyword">if</span> (n &lt; <span class="hljs-number">0</span>) {
    <span class="hljs-keyword">return</span> <span class="hljs-number">1</span>;
  } <span class="hljs-keyword">else</span> {
    <span class="hljs-keyword">return</span> n * calculateFactorial(n - <span class="hljs-number">1</span>);
  }
}

app.get(<span class="hljs-string">"/"</span>, <span class="hljs-function">(<span class="hljs-params">req, res</span>) =&gt;</span> {
  res.send(<span class="hljs-string">"Welcome to the Factorial Calculator"</span>);
});

app.get(<span class="hljs-string">"/factorial/:number"</span>, <span class="hljs-function">(<span class="hljs-params">req, res</span>) =&gt;</span> {
  <span class="hljs-keyword">const</span> number = <span class="hljs-built_in">parseInt</span>(req.params.number);
  <span class="hljs-keyword">const</span> factorial = calculateFactorial(number);
  res.send(<span class="hljs-string">`The factorial of <span class="hljs-subst">${number}</span> is: <span class="hljs-subst">${factorial}</span>`</span>);
});

app.listen(port, <span class="hljs-function">() =&gt;</span> {
  <span class="hljs-built_in">console</span>.log(<span class="hljs-string">`Server is listening on port <span class="hljs-subst">${port}</span>`</span>);
});
</code></pre>
<p>If you are an experienced developer, you must have already spotted the bug. But let's forget about it and go through the thought process together.</p>
<p>Having taken a brief look at the code, everything seems to be working absolutely fine. However, when we go to the <code>/factorial/:id</code> endpoint and enter any number, the output always appears to be 0.</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1690337333517/ecc7eddd-1352-432e-95e1-fe2010c1724b.png" alt class="image--center mx-auto" /></p>
<p>Let's find out the real reason for it. Set a breakpoint at <mark>line 19</mark> and step over each line. We can observe that the <code>number</code> variable correctly parses as an integer, but the <code>factorial</code> variable always seems to store the value 0. This is not the intended behavior, and the problem seems to lie in the <code>calculateFactorial</code> function.</p>
<p>Now that we are sure that the <code>calculateFactorial</code> function is the culprit, let's set up a breakpoint at <mark>line 20</mark> and <code>step into</code> the function. Go through each recursion step and notice that everything seems to work fine until the value of n is greater than 0. However, as soon as the value of <code>n == 0</code>, the function returns the value 0.</p>
<p>Ahh, I forgot to handle the case when the value of <code>'n' is 0</code>. That's why, when there is any value stored in the <code>number</code> variable, it was getting multiplied by zero, resulting in the overall value being 0.</p>
<p>Modify the function to also handle the <code>n == 0</code> case. The <code>calculateFactorial</code> code should now look like this.</p>
<pre><code class="lang-javascript"><span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-title">calculateFactorial</span>(<span class="hljs-params">n</span>) </span>{
  <span class="hljs-keyword">if</span> (n == <span class="hljs-number">0</span>) {
    <span class="hljs-keyword">return</span> <span class="hljs-number">1</span>;
  } <span class="hljs-keyword">else</span> {
    <span class="hljs-keyword">return</span> n * calculateFactorial(n - <span class="hljs-number">1</span>);
  }
}
</code></pre>
<p>Now, everything should work as expected🤞. Voilà!</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1690339232149/da8138a1-7ce9-481c-b3a0-523c070025d3.png" alt class="image--center mx-auto" /></p>
<p>I know this was a pretty simple and silly example. Even though it was a simple example, we still learned how to debug Node.js and applied it in real-world code.</p>
<div class="embed-wrapper"><div class="embed-loading"><div class="loadingRow"></div><div class="loadingRow"></div></div><a class="embed-card" href="https://media.giphy.com/media/BPJmthQ3YRwD6QqcVD/giphy.gif">https://media.giphy.com/media/BPJmthQ3YRwD6QqcVD/giphy.gif</a></div>
<p> </p>
<h2 id="heading-wrap-up">Wrap Up!</h2>
<p>In conclusion, becoming a Node.js troubleshooting pro will boost your programming skills. With debugging tools and techniques like these, you can fix bugs efficiently.</p>
<blockquote>
<p><strong>Embrace debugging, enjoy coding!</strong></p>
</blockquote>
<p>Follow me on LinkedIn 🌍: https://linkedin.com/in/shrijal007</p>
<p>Follow me on Github 👨‍💻: https://github.com/shricodev</p>
]]></content:encoded></item><item><title><![CDATA[Build your Portfolio with React and Tailwind in minutes]]></title><description><![CDATA[Discover how to create a sleek and modern portfolio using React and Tailwind CSS, and even learn how to host it for free on a .tech domain for a year.]]></description><link>https://shricodev.hashnode.dev/build-portfolio-with-react-and-tailwind</link><guid isPermaLink="true">https://shricodev.hashnode.dev/build-portfolio-with-react-and-tailwind</guid><category><![CDATA[WeMakeDevs]]></category><category><![CDATA[Web Development]]></category><category><![CDATA[React]]></category><category><![CDATA[Tailwind CSS]]></category><dc:creator><![CDATA[Shrijal Acharya]]></dc:creator><pubDate>Thu, 18 May 2023 06:27:39 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1684306706881/e58effa4-3a81-42b2-8a74-72eb07787740.webp" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In this article, I will share how you can build a Portfolio with React and Tailwind CSS, and host it for free on a <code>.tech</code> domain for free for a year with GitHub Student Developer Pack.</p>
<p>I choose to start learning React a few weeks ago. I was familiar with Python, Java, and Shell Script, but I was ignorant of front-end frameworks. So, I decided to give React a try. Will of contributing to various Frontend OpenSource Projects was another motivation for studying React. For this, I decided to take a free YouTube course on React by Bob Ziroll on <a target="_blank" href="https://www.youtube.com/watch?v=bMknfKXIFA8">freeCodeCamp</a>. I've posted my code for all the mini-projects <a target="_blank" href="https://github.com/shricodev/react-mini-playground">here</a>.</p>
<p>After finishing the tutorial, I was pretty confident with the React concepts and decided to do a major project. I had a thought: why not create a strong portfolio website? I already had a crappy <a target="_blank" href="https://shricodev.github.io/">portfolio</a> created using plain HTML, CSS, and JS.</p>
<p>Let's get started!🚀</p>
<h2 id="heading-setting-up-the-environment">Setting up the Environment</h2>
<p>We will use Yarn as the package manager. To Install Yarn, run the command:</p>
<pre><code class="lang-bash">npm install --global yarn
</code></pre>
<p>For setting up the local development environment, we will use Vite. Learn more about Vite <a target="_blank" href="https://vitejs.dev/">here</a>. To use Vite, run the following command:</p>
<pre><code class="lang-bash">yarn create vite
</code></pre>
<p>Give the project an appropriate project name, choose the framework, and hit enter. Now this should create an empty React project in your directory.</p>
<p>Inside the project folder run this command to install Vite as a development dependency.</p>
<pre><code class="lang-bash">yarn add vite --dev
</code></pre>
<p>Now, it's time to set up Tailwind CSS. Run this command to install Tailwind CSS and its dependencies.</p>
<pre><code class="lang-bash">yarn add --dev tailwindcss autoprefixer postcss
</code></pre>
<p>Now, we need to install Tailwind CSS config files. They help us to interact with our project and customize everything.</p>
<pre><code class="lang-bash">npx tailwindcss init -p
</code></pre>
<p>This command generates <code>tailwind.config.js</code> and<code>postcss.config.js</code> configuration files. Now for Tailwind CSS to work with the project, we need to specify the files it should look out for. Add <code>"./index.html", "./src/**/*.{js,ts,jsx,tsx}"</code> inside the content array in the <code>tailwind.config.js</code> file.</p>
<p>The <code>tailwind.config.js</code> the file should now look something like this:</p>
<pre><code class="lang-typescript"><span class="hljs-comment">/** @type {import('tailwindcss').Config} */</span>
<span class="hljs-built_in">module</span>.<span class="hljs-built_in">exports</span> = {
    content: [<span class="hljs-string">"./index.html"</span>, <span class="hljs-string">"./src/**/*.{js,ts,jsx,tsx}"</span>],
    theme: {
        extend: {},
    },
    plugins: [],
};
</code></pre>
<p>Now, the final step in setting up Tailwind CSS is to add its directive to the CSS file which is <code>src/index.css</code>. Clear all the default styling and add the following</p>
<pre><code class="lang-css"><span class="hljs-keyword">@tailwind</span> base;
<span class="hljs-keyword">@tailwind</span> components;
<span class="hljs-keyword">@tailwind</span> utilities;
</code></pre>
<p>Finally, run this command to start the Vite local development server:</p>
<pre><code class="lang-bash">yarn run dev
</code></pre>
<p>The terminal output should look something like this:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1684326833924/bfd8deba-10dc-45c5-8770-53b87e1e245f.png" alt class="image--center mx-auto" /></p>
<p>Hold the <strong>ctrl</strong> key and click on the link in the terminal. It should open a new tab that looks like the following:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1684328957812/ca6cbc2a-62f2-404c-b81b-7bf18443c145.png" alt="First look of the site" class="image--center mx-auto" /></p>
<p>Head on to the <code>App.tsx/jsx</code> file and remove the child divs of the div with the class name <code>App</code>. Insert the following in place of it.</p>
<pre><code class="lang-typescript">&lt;h1 className=<span class="hljs-string">"text-3xl font-bold underline text-center"</span>&gt;Hello world!&lt;/h1&gt;
</code></pre>
<p>This is the standard style of writing Tailwind CSS. Revisit the page and you should see this:</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1684333056136/ae273c31-ce46-4bb2-b91b-fe5afe045374.png" alt class="image--right mx-auto mr-0" /></p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1684333782537/1edea8df-da4b-4787-8d88-f128ef956af6.gif" alt class="image--center mx-auto" /></p>
<p>Congratulations on setting up your development server!<strong>🥳</strong></p>
<h2 id="heading-github-student-developer-pack-and-code">GitHub Student Developer Pack and Code</h2>
<p>It's not possible to explain all the code in this article. I've taken some UI references from a few YouTube videos. I will share the link to the Source Code <a target="_blank" href="https://github.com/shricodev/my-react-portfolio">here</a>.</p>
<p>Step by Step instructions on how to get GitHub Student Developer Pack is linked <a target="_blank" href="https://techsathi.com/github-student-developer-pack-how-to-apply">here</a>.</p>
<h2 id="heading-hosting">Hosting</h2>
<p>After you've gotten your GH Student Developer Pack, go to <a target="_blank" href="https://get.tech">get.tech</a>. Sign up with your GitHub Account and you should be able to claim your one-year free <code>.tech</code> domain plan.</p>
<p>Now, to host your Source Files, go to <a target="_blank" href="https://netlify.com">netlify.com</a> and create an account. Now, get back to the project folder and run the command:</p>
<pre><code class="lang-typescript">yarn run build
</code></pre>
<p>After running this command, you should have a <code>/dist</code> directory at the root of your project. The previous command is simply intended to give you the impression that this is the only folder that contains your deployment-related files. All this is handled by Netlify itself.</p>
<blockquote>
<p>💡Before, deploying the project make sure you push your project code to GitHub which will help in the rebuild of the code when you make any changes to the code in the future. So, you do not need to manually copy and paste your project each time in <a target="_blank" href="https://netlify.com">netlify.com</a></p>
</blockquote>
<p>Head on to Netlify and click the following:</p>
<p><code>"Add New Site" &gt; "Import an Existing Project" &gt; "Choose Github" &gt; "Select the repo"</code></p>
<p>For "<code>build settings"</code>, have these values</p>
<p><img src="https://cdn.hashnode.com/res/hashnode/image/upload/v1684344872776/c90ffe09-49f0-4e4e-a272-0402e1cc26b9.png" alt class="image--center mx-auto" /></p>
<p>Finally, click on <code>"Deploy Site"</code> and you should have your site deployed on <code>something.netlify.app</code>.</p>
<p>Now, to map this site to the domain we just purchased on <a target="_blank" href="https://get.tech">get.tech</a>, make sure you change the nameserver of your domain to Netlify nameservers. For this go under <code>"Options" &gt; "Set Up Netlify DNS"</code> and copy the nameservers of Netlify, and paste it in <code>"Custom nameserver"</code> inside <a target="_blank" href="https://get.tech">get.tech</a> domain settings. Now, in Netlify under <code>"Domain Settings" &gt; "Add custom domain"</code>, add your purchased domain name.</p>
<p>With these changes, you should now have your site deployed on your custom domain.🚀</p>
<h2 id="heading-tldr">TLDR;</h2>
<p>In conclusion, React and Vite go very smoothly together and can be used to build some very powerful front-end projects. In this blog, we discussed how to set up a React project with Vite locally, set up Tailwind CSS, get Github Developer Student Pack, and host the portfolio site for free.</p>
<p>It's recommended to maintain a proper folder structure in React by creating individual folders for each component. This practice increases the maintainability and scalability of your application, allowing for easier navigation and updates.</p>
<ul>
<li><p>Source Code for my Portfolio Site: <a target="_blank" href="https://github.com/shricodev/my-react-portfolio">Source Code</a></p>
</li>
<li><p>Visit my Portfolio: <a target="_blank" href="https://shricodev.tech">shricodev.tech</a></p>
</li>
</ul>
<h2 id="heading-lets-connect">Let's Connect</h2>
<p>If you enjoyed this post and would like to stay updated, feel free to connect with me on social media.</p>
<ul>
<li><p>Github: <a target="_blank" href="https://github.com/shricodev">https://github.com/shricodev</a></p>
</li>
<li><p>LinkedIn: <a target="_blank" href="https://linkedin.com/in/shrijal007">https://linkedin.com/in/shrijal007</a></p>
</li>
<li><p>Twitter: <a target="_blank" href="https://twitter.com/shricodev">https://twitter.com/shricodev</a></p>
</li>
</ul>
]]></content:encoded></item></channel></rss>