Bring Ted to life
Next Steps
Transcription:
- add automatic transcription (2)
- dynamic audio recording (ends after 3s of no speech found) (3)
- use llm context window/memary to store transcription & responses for context (clear graph for every new conversation) (9)
Model:
- change to llama 3 8B (1)
Server:
- Openvoice added to ec2 instance (5)
- benchmark ollama vs server response times (8)
Speech:
- openvoice over Ted's voice (4)
- speaker integration (6)
Design:
-
redesign & print 3d case (7)
-
Build Teddy (insert mcu & speaker) (10)
-
Documentation (11)
Done:
- CConnect mcu & battery (hardware)
- Gather data (software)
- transcribe using mcu
- host llama 2 on ec2 instance
![Screenshot 2024-04-08 at 8 59 01 PM](https://private-user-images.githubusercontent.com/120517860/320697480-11a49c6b-f658-4744-a2a8-c5978cb8125d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk3NDgwLTExYTQ5YzZiLWY2NTgtNDc0NC1hMmE4LWM1OTc4Y2I4MTI1ZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zNGI4MTEwZDMzYTQxZjVjZWE0ZGU3ODJkY2FlYzBjNzQzZmE3M2Q2ZGMyNDExYWIyMzhlNjBjODcxYmZiOTdkJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.ODgzfGa7XEpF2j2dnAZ1G7vhZSREN-eerMEqqgtD6u0)
![Screenshot 2024-04-08 at 8 59 15 PM](https://private-user-images.githubusercontent.com/120517860/320697543-210ad940-f333-44ec-950d-702f0b0c0c37.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk3NTQzLTIxMGFkOTQwLWYzMzMtNDRlYy05NTBkLTcwMmYwYjBjMGMzNy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1kYzE1ZGIzNzk3ZWFlNjAwMjZmM2Y4OTdlMGIyMjgwNmQyYTk2YTE2ZjdlNjgyYzJlNmM5ZDNjNzgxZjFmN2Q0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.uia7j0irDuEF_dKtxw0mPeDfxOzu94gJWVZCwB4XUnA)
![Screenshot 2024-04-08 at 8 59 26 PM](https://private-user-images.githubusercontent.com/120517860/320697579-7189064c-59de-4cbc-932f-8147e65f8f71.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk3NTc5LTcxODkwNjRjLTU5ZGUtNGNiYy05MzJmLTgxNDdlNjVmOGY3MS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02MmQyMGJkYTkwMDc3M2QxNGEwMWZjYzQ1ZGJkZTM3NGQ2Yzc2ZWQ0Njk1N2FiODQyMzY2MTc1OGU0MzZiMzljJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.CjpPbxZQogZjASwEbWCT7Rpr8kN5kSDycTB8_lGlcro)
![Screenshot 2024-04-08 at 8 59 44 PM](https://private-user-images.githubusercontent.com/120517860/320697611-7fe7f5b6-ba90-4622-b626-b58fac5eb2de.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk3NjExLTdmZTdmNWI2LWJhOTAtNDYyMi1iNjI2LWI1OGZhYzVlYjJkZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wYzM1NmJmY2Y3NDA5MmNlYWE3NGVmZWNmMjYwZDMzYzFkOGU3NTdkNzljMjJiMGQxYTJmZmI2YTczNDQ2ZTlhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.WKmpTVQ4Y8KMqSVWfE2K1_aine4ynh7HpgX88cgPtIs)
![Screenshot 2024-04-08 at 9 11 05 PM](https://private-user-images.githubusercontent.com/120517860/320699361-02fb63bb-c964-431a-aa61-6eb7f99762dc.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk5MzYxLTAyZmI2M2JiLWM5NjQtNDMxYS1hYTYxLTZlYjdmOTk3NjJkYy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wNDI1NDg3YjVlYmMwZGM4NmE5ZDEwMmM1NDE2NDNkYjAyNDllNjNjZTc0YTE0MTljNGU1OTM3Mzg4NjY4MWU1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.PpzHFXX9ytoGvIY6kCW7E2BHKBogrt7XIwR8MWlOy9s)
![Screenshot 2024-04-08 at 9 03 35 PM](https://private-user-images.githubusercontent.com/120517860/320698173-d7d8b5a7-0516-4e04-995b-5ade79234901.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk4MTczLWQ3ZDhiNWE3LTA1MTYtNGUwNC05OTViLTVhZGU3OTIzNDkwMS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yMjc0YTQwMzZiZjg2OTc0N2UyNWUzMjU5ZTE4YjA5YmY5MjBmM2RhNjk0NzY1MzcyNDdkNTRkOGEwZTRjMDcxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.FE7A6eo5AS1AEjEq3WGTTLiDerR7u4ceH3hq3raixPM)
![Screenshot 2024-04-08 at 9 03 53 PM](https://private-user-images.githubusercontent.com/120517860/320698216-cf1547c7-4a30-49c8-9d79-6ead48e34aab.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIwNjk4MjE2LWNmMTU0N2M3LTRhMzAtNDljOC05ZDc5LTZlYWQ0OGUzNGFhYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0yYWUwOTc2OTJlNDg3ZjBiYWRiNDgyN2I5ZGViMmQ0Y2E0MjI3MDkwMzQwOWMyMzE5OGJkNDI0ZjFjZDkyY2JhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.jdFi8Zk4bV0ha-Oc5LxGGMWJMLP4gP1nZVAviXIHAiY)
![Screenshot 2024-04-14 at 12 40 46 PM](https://private-user-images.githubusercontent.com/120517860/322292319-ba669553-1896-47cd-97ea-6318195866b9.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MzkwMTYxNDEsIm5iZiI6MTczOTAxNTg0MSwicGF0aCI6Ii8xMjA1MTc4NjAvMzIyMjkyMzE5LWJhNjY5NTUzLTE4OTYtNDdjZC05N2VhLTYzMTgxOTU4NjZiOS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjUwMjA4JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI1MDIwOFQxMTU3MjFaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05MjE4YThhZTg0MzdmM2Q3Y2U1M2Y0ZTdmY2ZmY2RkYWM0NGU4MTE3NjE3YjMxNzYzNjgyOTVhNWRiYjZhMmYxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.ilDrQeNk-d1Movex8H2-q4SywZs-VjH9_M-9FQ7TKZE)