Adding code to transcribe long audio files using streaming analyze content #13361
Conversation
product-auto-label
bot
added
samples
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hello @amanuppal17208, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
This pull request introduces a new Python script, analyze_content_stream.py, to transcribe long audio files using the Dialogflow API's streaming analyze content feature. The script splits the audio into smaller chunks to handle large files, then sends each chunk to the Dialogflow API for transcription. The pydub library is added to handle audio splitting.
Highlights
- New Feature: Adds a script to transcribe long audio files using Dialogflow's streaming analyze content API.
- Audio Chunking: Implements audio splitting to handle large audio files within the API's constraints.
- Dependency: Introduces
pydubas a dependency for audio processing.
Changelog
- dialogflow/analyze_content_stream.py
- Adds a new script to transcribe long audio files using Dialogflow's streaming analyze content API.
- Implements audio splitting using
pydubto handle large audio files. - Creates a conversation and participant for each audio chunk.
- Streams audio chunks to Dialogflow for transcription.
- Concatenates the transcriptions from each chunk to produce a final transcript.
- dialogflow/requirements.txt
- Adds
pydub==0.25.1as a new dependency.
- Adds
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
A stream of sound,
To text it is bound,
Dialogflow's ear hears.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
The pull request introduces a new script, analyze_content_stream.py, to transcribe long audio files using the Dialogflow API's streaming analyze content feature. The script splits the audio into chunks, processes each chunk using the API, and concatenates the results. Overall, the code is well-structured and addresses the intended functionality. However, there are areas where improvements can be made to enhance readability, error handling, and efficiency.
Summary of Findings
- Error Handling: The code lacks explicit error handling for API calls and file operations. Adding error handling would make the script more robust.
- Resource Management: The temporary audio chunks created by
split_audioare not deleted after processing, which can lead to unnecessary disk usage. Consider deleting these files after they are no longer needed. - Clarity of Variable Names: Some variable names, like
outinrun_analyze_content_streaming, could be more descriptive to improve code readability. - String concatenation: The string concatenation in the final transcript can be made more efficient by using
join.
Merge Readiness
The pull request introduces valuable functionality for transcribing long audio files. However, before merging, it's recommended to address the identified issues, particularly regarding error handling and resource management. Addressing these points will improve the script's robustness and maintainability. I am unable to directly approve this pull request, and other reviewers should also review this code before merging.
There was a problem hiding this comment.
Thank you for your first contribution to Cloud Samples!
There are some fundamental issues with this PR. Apart from the comments below, if you can update the PR description with a link to the internal bug this relates to, we can help get this sample onboarded (including tests, usage including any region tags, and testing resources) (I could not get this sample working locally)
| Also please contact Google to get credentials of this project and set up the | ||
| credential file json locations by running: | ||
| export GOOGLE_APPLICATION_CREDENTIALS=<cred_json_file_location> | ||
|
|
||
| Example usage: | ||
| export GOOGLE_CLOUD_PROJECT='cloud-contact-center-ext-demo' | ||
| export CONVERSATION_PROFILE='FnuBYO8eTBWM8ep1i-eOng' | ||
| export GOOGLE_APPLICATION_CREDENTIALS='/Users/ruogu/Desktop/keys/cloud-contact-center-ext-demo-78798f9f9254.json' | ||
| export AUDIO_FILE_PATH='/Users/ruogu/Desktop/book_a_room.wav' | ||
|
|
||
| python analyze_content_stream.py |
There was a problem hiding this comment.
I understand this might have been copied from another sample, but this is not valid for any other user that isn't you/the original author.
| @@ -0,0 +1,114 @@ | |||
| # Copyright 2025 Google LLC | |||
There was a problem hiding this comment.
This entire sample may supposed to live in dialogflow-cx rather than dialogflow; the samples in dialogflow have not been edited in a while, and are currently untested.
| @@ -0,0 +1,114 @@ | |||
| # Copyright 2025 Google LLC | |||
There was a problem hiding this comment.
This file also matches an existing test file for a (potentially) unrelated sample.
Description
Fixes #
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist
nox -s py-3.9(see Test Environment Setup)nox -s lint(see Test Environment Setup)