Intro to Rust for JS developers - Part 5

Repo

Welcome back to this extra mile to polish our CLI. As mentioned in the previous article, I want to implement two features to enhance our app:

  • A safeguard to abort an API call if the payload exceeds 4096. This aims to prevents extra charges for invalid requests.
  • Install the app globally to be able to run it from anywhere using the terminal.

1. Safeguard

The logic for the safeguard is quite simple. We add a method is_text_within_limit to OpenAIClient that uses built-in functions .chars().count() to check the size of the payload. If it's more than 4096, just panic!


const API_LIMIT: usize = 4096;

pub struct OpenAIClient {
    url: String,
    headers: HeaderMap,
    http: Client,
    pub payload_limit: usize,
}

impl OpenAIClient { 
	 ...
	
	fn is_text_within_limit(&self, text: &str) -> bool {
     text.chars().count() <= self.payload_limit
  }

	pub async fn text_to_speech(&self, text: String, output_path: &str) {
        if !self.is_text_within_limit(&text) {
            panic!(
                "Text exceeds the maximum allowed character count of {}",
                self.payload_limit
            );
        }

        let request_body = json!({
            "model": "tts-1",
            "input": text,
            "voice": "alloy"
        });

		// ...

	}
}

Now we need to update the processing to get the limit from the client:

fn split_into_chunks(text: &str, limit: usize) -> Vec<String> {
    text.chars()
        .collect::<Vec<char>>()
        .chunks(limit)
        .map(|chunk| chunk.iter().collect::<String>())
        .collect()
}

pub async fn audiofy(text: String, output_path: &str) {
    println!("🎤 Audiofy...");

    let client = OpenAIClient::new();
    let text_chunks = split_into_chunks(&text, client.payload_limit);
   
   // ...
}

Well done! Now we are sure to never send (and being charged for) an invalid request because for a large payload.

2. Release

cargo install --path .

That's it! pretty simple 😀 ?! Rust has this handy command that's going to make our lives easier. It tells Cargo to install our project directly from the current directory. So, instead of jumping through hoops every time we want to run our CLI, we use this command once, and voilà, we can launch our app from anywhere in our terminal just like any other command line tool.

Open your terminal and run :

audiofy https://www.sadry.dev/articles/intro-to-rust-for-js-devs-part-1

The command will be executed but you will run into an issue: OPENAI_API_KEY not found!: NotPresent.

Indeed our environment variables are not added to our program. They are available in the source code only. We need to export them in the terminal:

export OPENAI_ORG_ID="YOUR_ORG_ID"
export OPENAI_API_KEY="YOUR_API_KEY"
export OPENAI_API_URL="https://api.openai.com/v1/audio/speech"

After resolving the initial problem, a new issue arises when attempting to run the command again: it reports that folder creation is not allowed. Recall that we had created an output folder at the root of our codebase to store the MP3 files returned by the API. However, now that our command can be executed from anywhere in the terminal, the original path becomes invalid, and we can no longer assume we have the necessary permissions to access it.

To overcome this issue, it's essential to save the generated files in a location that's always accessible, regardless of where the command is initiated. The user's home directory seems to be appropriate for this use case.

Let's update our code to move the output folder to the home directory. We need to make sure that our approach is valid across different platforms. For this purpose, we'll utilize the dirs crate. The dirs crate is a simple library that provides an easy way to access commonly used directories on various platforms, such as the home directory, cache directory, executable directory,… There is a sudle difference though to use it in our codebase, it uses an old syntax extern crate dirs;.

Before the 2018 edition of Rust, you needed to declare external crates explicitly in your root file (usually main.rs or lib.rs) to let the compiler know you intended to use them. This declaration was done with the extern crate dirs; syntax. It was a requirement for the compiler to understand that dirs was not a module within your package but rather an external crate that needed to be linked.

Today, you directly use use statements to bring items from external crates into scope, similar to how you would with modules within your own crate. However, for compatibility with older code or in certain edge cases where the automatic crate import doesn't work as expected, you might still need extern crate.

// audio/audiofy.rs

// ...
use std::fs::{create_dir_all, read_dir, remove_dir_all};
extern crate dirs;


pub async fn audiofy(text: String, output_path: &str) {
    println!("🎤 Audiofy...");

    let client = OpenAIClient::new();
    let text_chunks = split_into_chunks(&text, client.payload_limit);

    let user_home_dir = dirs::home_dir().expect("Unable to get home directory"); 
    
    let temp_path = format!("{}/{}", OUTPUT_DIR, TEMP_DIR);
    let absolute_temp_path = user_home_dir.join(temp_path);
    let absolute_temp_path_str = absolute_temp_path.to_str().unwrap();
    
    create_dir_all(&absolute_temp_path).expect("Failed to create tmp directory");

    println!("=> {} chunks to transform:", text_chunks.len());

    for (i, chunk) in text_chunks.iter().enumerate() {
        println!("Processing chunk at index {}...", i);
        let chunk_path = format!("{}/chunk_{}.mp3", absolute_temp_path_str, i);
        client.text_to_speech(chunk.to_string(), &chunk_path).await;
    }

    let absolute_outrput_path = user_home_dir.join(output_path);
    let absolute_outrput_path_str = absolute_outrput_path.to_str().unwrap();
    
    merge_audio_chunks(&absolute_temp_path_str, &absolute_outrput_path_str);
    remove_dir_all(&absolute_temp_path_str).expect("Failed to remove directory");
}

dirs::home_dir() returns the path to the home directory as PathBuf object that represents a mutable path on the filesystem. The .join(temp_path) method appends the temp_path (created in the first step) to user_home_dir, creating a new PathBuf instance that represents the absolute path to the temporary directory within the user's home directory.

Then we convert absolute_temp_path from a PathBuf into a string slice (&str) to be able to pass it as argument to create_dir_all. This function attempts to create the directory along with all necessary parent directories if they don't already exist.

Thats is pretty much it. Now our CLI should be functioning as intended. Open your terminal and run:

audiofy https://www.sadry.dev/articles/intro-to-rust-for-js-devs-part-1

Enjoy your podcast 🔊 🎧

3. The End

As we wrap up our series of tutorials on building a CLI to turn articles into podcasts, it's clear our approach has been quite simple. Right now, we only take the article's content by looking for an article, which is very basic.
A better way would involve using more complex HTML parsing to better understand and use the structure of the articles, which would improve how we handle code snippets and describe pictures.

Additionally, the code implemented does not reflect the best practices among Rust community. There's a lot more for me to learn, including special Rust features like lifetimes and smart pointers. However, I'm really happy with what I've learned so far. This project taught me about important programming concepts like the call stack and heap memory, which I had heard about but never really looked into before. Now, I feel comfortable navigating a Rust codebase and keeping up with JS tooling made in Rust.

I hope you found this series helpful and learned from it, just as much as I did.