Skip to content
/ RealChar Public

🎙️🤖Create, Customize and Talk to your AI Character/Companion in Realtime (All in One Codebase!). Have a natural seamless conversation with AI everywhere (mobile, web and terminal) using LLM OpenAI GPT3.5/4, Anthropic Claude2, Chroma Vector DB, Whisper Speech2Text, ElevenLabs Text2Speech🎙️🤖

realchar.ai/

License

MIT license
2.4k stars 246 forks
Star
Notifications

Shaunwei/RealChar

main
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
47 branches 16 tags
Code

Latest commit

@pycui
pycui Merge pull request #118 from ezioruan/main
4c8c872 Jul 19, 2023
Merge pull request #118 from ezioruan/main 
Update README.md
4c8c872

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
.github/workflows
Move to self-hosted
July 15, 2023 02:20
alembic
Add session IDs to table
July 18, 2023 22:53
client
Merge pull request #95 from Shaunwei/pcui/change-client_id
July 18, 2023 01:41
realtime_ai_character
Add session IDs to table
July 18, 2023 22:53
sample_cloud_deployment
Notes
July 17, 2023 01:48
.env.example
Merge branch 'main' into main
July 18, 2023 17:25
.gitignore
new frontend
July 10, 2023 19:52
Dockerfile
Optimize docker build
July 18, 2023 11:28
LICENSE
Initial commit
June 26, 2023 13:51
README.md
Merge pull request #118 from ezioruan/main
July 19, 2023 10:37
alembic.ini
Init
June 26, 2023 17:32
cli.py
Update CLI default host to avoid confusion around 0.0.0.0
July 18, 2023 11:07
contribute.md
Add contribute guide
July 13, 2023 21:31
entrypoint.sh
rename and chmod
July 16, 2023 01:33
requirements.txt
Fix chromadb version
July 18, 2023 01:17
RealChar. - Your Realtime AI Character ✨ Demo Demo 1 - with AI Elon about cage fight! Demo 2 - with AI Raiden about AI and "real" memory 🎯 Key Features 🔬 Tech stack 📚 Comparison with existing products 👨‍🚀 Prerequisites 0. Setup Tutorial 1. LLM - OpenAI API Token 1.1 (Optional) Prepare LLM - Anthropic(Claude 2) API Token 2. (Optional) Prepare Speech to Text - Google Cloud API 3. Prepare Text to Speech - ElevenLabs API Key 💿 Installation via Python (Optional) 📀 Installation via Docker 🆕! LangSmith integration ⭐️ Star History 📍 Roadmap 🫶 Contribute to RealChar 🎲 Community

README.md

RealChar. - Your Realtime AI Character


RealChar-logo

🎙️🤖Create, customize and talk to your AI Character/Companion in realtime🎙️🤖

✨ Demo

Try our site at RealChar.ai

Demo 1 - with AI Elon about cage fight!

elon-edit-camera.mp4

Demo 2 - with AI Raiden about AI and "real" memory

raiden.mp4

Demo settings: Web, GPT4, ElevenLabs with voice clone, Chroma, Google Speech to Text

🎯 Key Features

  • Easy to use: No coding required to create your own AI character.
  • Customizable: You can customize your AI character's personality, background, and even voice
  • Realtime: Talk to or message your AI character in realtime
  • Multi-Platform: You can talk to your AI character on web, terminal and mobile(Yes. we open source our mobile app)
  • Most up-to-date AI: We use the most up-to-date AI technology to power your AI character, including OpenAI, Anthropic Claude 2, Chroma, Whisper, ElevenLabs, etc.
  • Modular: You can easily swap out different modules to customize your flow. Less opinionated, more flexible. Great project to start your AI Engineering journey.

🔬 Tech stack

RealChar-tech-stack

📚 Comparison with existing products

👨‍🚀 Prerequisites

Before you begin setting up this project, please ensure you have completed the following tasks:

0. Setup Tutorial

1. LLM - OpenAI API Token

👇click me This application utilizes the OpenAI API to access its powerful language model capabilities. In order to use the OpenAI API, you will need to obtain an API token.

To get your OpenAI API token, follow these steps:

  1. Go to the OpenAI website and sign up for an account if you haven't already.
  2. Once you're logged in, navigate to the API keys page.
  3. Generate a new API key by clicking on the "Create API Key" button.
  4. Copy the API key and store it safely.
  5. Add the API key to your environment variable, e.g. export OPENAI_API_KEY=<your API key>

(Optional) To use Azure OpenAI API instead, refer to the following section:

  1. Set API type export OPENAI_API_TYPE=azure

If you want to use the earlier version 2023-03-15-preview:

export OPENAI_API_VERSION=2023-03-15-preview

  1. To set the base URL for your Azure OpenAI resource. You can find this in the Azure portal under your Azure OpenAI resource.

export OPENAI_API_BASE=https://your-base-url.openai.azure.com

  1. To set the OpenAI model deployment name for your Azure OpenAI resource.

export OPENAI_API_MODEL_DEPLOYMENT_NAME=gpt-35-turbo-16k

  1. To set the OpenAIEmbeddings model deployment name for your Azure OpenAI resource.

export OPENAI_API_EMBEDDING_DEPLOYMENT_NAME=text-embedding-ada-002

1.1 (Optional) Prepare LLM - Anthropic(Claude 2) API Token

👇click me

To get your Anthropic API token, follow these steps:

  1. Go to the Anthropic website and sign up for an account if you haven't already.
  2. Once you're logged in, navigate to the API keys page.
  3. Generate a new API key by clicking on the "Create Key" button.
  4. Copy the API key and store it safely.
  5. Add the API key to your environment variable, e.g. export ANTHROPIC_API_KEY=<your API key>

2. (Optional) Prepare Speech to Text - Google Cloud API

👇click me

To get your Google Cloud API credentials.json, follow these steps:

  1. Go to the GCP website and sign up for an account if you haven't already.
  2. Follow the guide to create a project and enable Speech to Text API
  3. Put google_credentials.json in the root folder of this project. Check GCP website
  4. Change SPEECH_TO_TEXT_USE to use GOOGLE in your .env file

3. Prepare Text to Speech - ElevenLabs API Key

👇click me 1. Creating an ElevenLabs Account Visit [ElevenLabs](https://beta.elevenlabs.io/) to create an account. You'll need this to access the text to speech and voice cloning features.

  1. In your Profile Setting, you can get an API Key. Save it in a safe place.

  2. Set API key in your .env file:

ELEVEN_LABS_API_KEY=<api key>

💿 Installation via Python

  • Step 1. Clone the repo 
    git clone https://github.com/Shaunwei/RealChar.git && cd RealChar
  • Step 2. Install requirements 
    # for mac
brew install portaudio
brew install ffmpeg
    # for ubuntu
sudo apt update
sudo apt install portaudio19-dev
sudo apt install ffmpeg
    • Then install all python requirements
    pip install -r requirements.txt
  • Step 3. Create an empty sqlite database if you have not done so before 
    sqlite3 test.db "VACUUM;"
  • Step 4. Run db upgrade 
    alembic upgrade head
  • Step 5. Setup .env: update API keys and select module 
    cp .env.example .env
  • Step 6. Run server with cli.py or use uvicorn directly 
    python cli.py run-uvicorn
# or
uvicorn realtime_ai_character.main:app
  • Step 7. Run client:
    • Use GPT4 for better conversation and Wear headphone for best audio(avoid echo)
    • Web client: Open your web browser and navigate to http://localhost:8000 (NOT 0.0.0.0:8000)
    • (Optional) Terminal client: Run the following command in your terminal
    python client/cli.py
    • (Optional) mobile client: open client/mobile/ios/rac/rac.xcodeproj/project.pbxproj in Xcode and run the app
  • Step 8. Select one character to talk to, then start talking

(Optional) 📀 Installation via Docker

👇click me

  1. Docker image: you can use our docker image directly

    docker pull shaunly/real_char:latest

    (Or you want build yourself) Build docker image

    python cli.py docker-build

    If you have issues with docker (especially on a non-Linux machine), please refer to https://docs.docker.com/get-docker/ (installation) and https://docs.docker.com/desktop/troubleshoot/overview/ (troubleshooting).

  2. Run docker image with .env file

    python cli.py docker-run

  3. Go to http://localhost:8000 (NOT 0.0.0.0:8000) to start talking or use terminal client

    python client/cli.py

🆕! LangSmith integration

👇click me

If you have access to LangSmith, you can edit these environment variables to enable:

LANGCHAIN_TRACING_V2=false # default off
LANGCHAIN_ENDPOINT=https://api.smith.langchain.com
LANGCHAIN_API_KEY=YOUR_LANGCHAIN_API_KEY
LANGCHAIN_PROJECT=YOUR_LANGCHAIN_PROJECT

And it should work out of the box.


⭐️ Star History

Star History Chart

📍 Roadmap

  • Launch v0.0.0 and build a community
  • Add more characters
  • Move away from Vanilla JS
  • Launch mobile app
  • Add authentication

🫶 Contribute to RealChar

Please check out our Contribution Guide!

🎲 Community

About

🎙️🤖Create, Customize and Talk to your AI Character/Companion in Realtime (All in One Codebase!). Have a natural seamless conversation with AI everywhere (mobile, web and terminal) using LLM OpenAI GPT3.5/4, Anthropic Claude2, Chroma Vector DB, Whisper Speech2Text, ElevenLabs Text2Speech🎙️🤖

RealChar.ai/

Resources

Readme

License

MIT license

Stars

2.4k stars

Watchers

24 watching

Forks

246 forks
Report repository

Releases 15

v0.0.0-fix-thumb: Merge pull request #78 from eltociear/patch-1 Latest
Jul 16, 2023
+ 14 releases

Packages

No packages published

Contributors 11

Languages