Compare commits

...

No commits in common. 'main' and '0.2.5' have entirely different histories.
main ... 0.2.5

1
.gitattributes vendored

@ -1 +0,0 @@
vsc-dl-x64 export-ignore

133
.github/CODEOWNERS vendored

@ -1,133 +0,0 @@
# File: CODEOWNERS
* @Pythagora-io/maintainers
# File: .github/ip_assignment.yml
assign_to_owner: true
license: FSL-1.1-MIT
# File: .github/copyright_template.txt
Copyright (c) 2024 Pythagora Technologies Inc. All rights reserved.
This project is licensed under the terms of the FSL-1.1-MIT license.
# File: LICENSE
FSL-1.1-MIT License
Copyright (c) 2024 Pythagora Technologies Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
# Functional Source License, Version 1.1, MIT Future License
## Abbreviation
FSL-1.1-MIT
## Notice
Copyright 2024 Pythagora Technologies, Inc.
## Terms and Conditions
### Licensor ("We")
The party offering the Software under these Terms and Conditions.
### The Software
The "Software" is each version of the software that we make available under
these Terms and Conditions, as indicated by our inclusion of these Terms and
Conditions with the Software.
### License Grant
Subject to your compliance with this License Grant and the Patents,
Redistribution and Trademark clauses below, we hereby grant you the right to
use, copy, modify, create derivative works, publicly perform, publicly display
and redistribute the Software for any Permitted Purpose identified below.
### Permitted Purpose
A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
means making the Software available to others in a commercial product or
service that:
1. substitutes for the Software;
2. substitutes for any other product or service we offer using the Software
that exists as of the date we make the Software available; or
3. offers the same or substantially similar functionality as the Software.
Permitted Purposes specifically include using the Software:
1. for your internal use and access;
2. for non-commercial education;
3. for non-commercial research; and
4. in connection with professional services that you provide to a licensee
using the Software in accordance with these Terms and Conditions.
### Patents
To the extent your use for a Permitted Purpose would necessarily infringe our
patents, the license grant above includes a license under our patents. If you
make a claim against any party that the Software infringes or contributes to
the infringement of any patent, then your patent license to the Software ends
immediately.
### Redistribution
The Terms and Conditions apply to all copies, modifications and derivatives of
the Software.
If you redistribute any copies, modifications or derivatives of the Software,
you must include a copy of or a link to these Terms and Conditions and not
remove any copyright notices provided in or with the Software.
### Disclaimer
THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
### Trademarks
Except for displaying the License Details and identifying us as the origin of
the Software, you have no right under these Terms and Conditions to use our
trademarks, trade names, service marks or product names.
## Grant of Future License
We hereby irrevocably grant you an additional license to use the Software under
the MIT license that is effective on the second anniversary of the date we make
the Software available. On or after that date, you may use the Software under
the MIT license, in which case the following will apply:
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

@ -1,2 +0,0 @@
Copyright (c) 2024 Pythagora Technologies Inc. All rights reserved.
This project is licensed under the terms of the FSL-1.1-MIT license.

@ -1,2 +0,0 @@
assign_to_owner: true
license: FSL-1.1-MIT

@ -1,183 +0,0 @@
name: Staging Cloud Build and Deploy
on:
workflow_dispatch:
inputs:
extension_commit_sha:
description: "Extension commit SHA (defaults to latest commit on selected branch)"
required: false
type: string
default: ""
extension_branch:
description: "Extension branch to use"
required: false
type: string
default: "main"
core_commit_sha:
description: "Core commit SHA (defaults to latest commit on selected branch)"
required: false
type: string
default: ""
core_branch:
description: "Core branch to use"
required: false
type: string
default: "main"
permissions:
contents: write
jobs:
get-versions:
runs-on: ubuntu-latest
outputs:
extension_sha: ${{ steps.get-extension-sha.outputs.extension_sha }}
core_sha: ${{ steps.get-core-sha.outputs.core_sha }}
combined_tag: ${{ steps.generate-tag.outputs.combined_tag }}
steps:
- name: Checkout Extension
uses: actions/checkout@v4
with:
repository: Pythagora-io/gpt-pilot-vs-code-extension
token: ${{ secrets.GH_PAT }}
path: tmp_extension
ref: ${{ inputs.extension_commit_sha || inputs.extension_branch }}
fetch-depth: 0
- name: Get extension SHA
id: get-extension-sha
run: |
cd tmp_extension
SHA=$(git rev-parse HEAD)
echo "Extension SHA: $SHA"
echo "extension_sha=$SHA" >> $GITHUB_OUTPUT
cd ..
- name: Checkout Core
uses: actions/checkout@v4
with:
repository: Pythagora-io/pythagora-v1
token: ${{ secrets.GH_PAT }}
path: tmp_core
ref: ${{ inputs.core_commit_sha || inputs.core_branch }}
fetch-depth: 0
- name: Get core SHA
id: get-core-sha
run: |
cd tmp_core
SHA=$(git rev-parse HEAD)
echo "Core SHA: $SHA"
echo "core_sha=$SHA" >> $GITHUB_OUTPUT
cd ..
- name: Generate combined image tag
id: generate-tag
run: |
EXT_SHORT=$(echo "${{ steps.get-extension-sha.outputs.extension_sha }}" | cut -c1-7)
CORE_SHORT=$(echo "${{ steps.get-core-sha.outputs.core_sha }}" | cut -c1-7)
TAG="ext-${EXT_SHORT}_core-${CORE_SHORT}"
echo "Combined tag: $TAG"
echo "combined_tag=$TAG" >> $GITHUB_OUTPUT
deploy:
needs: get-versions
runs-on: ubuntu-latest
steps:
- name: Checkout Extension
uses: actions/checkout@v4
with:
repository: Pythagora-io/gpt-pilot-vs-code-extension
token: ${{ secrets.GH_PAT }}
path: tmp_extension
ref: ${{ needs.get-versions.outputs.extension_sha }} # Use SHA instead of version
fetch-depth: 0
- name: Checkout Core
uses: actions/checkout@v4
with:
repository: Pythagora-io/pythagora-v1
token: ${{ secrets.GH_PAT }}
path: tmp_core
ref: ${{ needs.get-versions.outputs.core_sha }}
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Setup extension
run: |
cd tmp_extension
npm install -g @vscode/vsce@3.3.2
npm run build
vsce package
cd ..
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Login to ECR
uses: docker/login-action@v3
with:
registry: ${{ secrets.ECR_URL }}
username: ${{ vars.AWS_ACCESS_KEY_ID }}
password: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
- name: Docker build and push
run: |
cd tmp_core
cp ../tmp_extension/*.vsix ./pythagora-vs-code.vsix
docker build --platform=linux/amd64 -f Dockerfile -t ${{ secrets.ECR_URL }}/pythagora/workspace:${{ needs.get-versions.outputs.combined_tag }} .
docker push ${{ secrets.ECR_URL }}/pythagora/workspace:${{ needs.get-versions.outputs.combined_tag }}
cd ..
- name: Checkout Workspace Helm
uses: actions/checkout@v4
with:
repository: Pythagora-io/workspace-helm
token: ${{ secrets.GH_PAT }}
path: tmp_wh
ref: 'staging'
fetch-depth: 0
- name: Install yq
run: |
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq
chmod +x /usr/local/bin/yq
- name: Commit and push changes
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
cd tmp_wh
new_value="${{ needs.get-versions.outputs.combined_tag }}"
file_path="chart/values.yaml"
current_value=$(yq '.workspace.image.tag' "$file_path")
if [ "$current_value" = "$new_value" ]; then
echo "No changes to commit, skipping..."
else
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
yq -i ".workspace.image.tag = \"${new_value}\"" "$file_path"
git add $file_path
VERSION=${{ needs.get-versions.outputs.combined_tag }}
git commit -m "version $VERSION"
git push origin staging
fi
cd ..

4
.gitignore vendored

@ -8,7 +8,6 @@ dist/
workspace/
pilot-env/
venv/
data/
.coverage
*.code-workspace
@ -16,10 +15,7 @@ data/
.env
*.pyc
*.db
*.db-shm
*.db-wal
config.json
poetry.lock
.DS_Store
*.log
pythagora-vs-code.vsix

@ -1,9 +0,0 @@
# (2025-08-25)
### Reverts
* Revert "Implemented weekend discount" ([734e0c7](https://github.com/Pythagora-io/pythagora-v1/commit/734e0c726b179a45f235a0fd230a6310c77ae740))

@ -1,99 +0,0 @@
# Use Ubuntu 22.04 as the base image with multi-arch support
FROM ubuntu:22.04
# Use buildx args for multi-arch support
ARG TARGETPLATFORM
ARG BUILDPLATFORM
# Set defaults for TARGETPLATFORM to ensure it's available in scripts
ENV TARGETPLATFORM=${TARGETPLATFORM:-linux/amd64}
# Copy VSIX file first
COPY pythagora-vs-code.vsix /var/init_data/pythagora-vs-code.vsix
# Install all dependencies
COPY cloud/setup-dependencies.sh /tmp/setup-dependencies.sh
RUN chmod +x /tmp/setup-dependencies.sh && \
/tmp/setup-dependencies.sh && \
rm /tmp/setup-dependencies.sh
ENV PYTH_INSTALL_DIR=/pythagora
# Set up work directory
WORKDIR ${PYTH_INSTALL_DIR}/pythagora-core
# Add Python requirements
ADD requirements.txt .
# Create and activate a virtual environment, then install dependencies
RUN python3 -m venv venv && \
. venv/bin/activate && \
pip install -r requirements.txt
# Copy application files
ADD main.py .
ADD core core
ADD pyproject.toml .
ADD cloud/config-docker.json config.json
# Set the virtual environment to be automatically activated
ENV VIRTUAL_ENV=${PYTH_INSTALL_DIR}/pythagora-core/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV PYTHAGORA_DATA_DIR=${PYTH_INSTALL_DIR}/pythagora-core/data/
RUN mkdir -p data
# Expose MongoDB and application ports
EXPOSE 27017 8000 8080 5173 3000
# Create a group and user
RUN groupadd -g 1000 devusergroup && \
useradd -m -u 1000 -g devusergroup -s /bin/bash devuser && \
echo "devuser:devuser" | chpasswd && \
adduser devuser sudo && \
echo "devuser ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
# Set up entrypoint and VS Code extension
ADD cloud/entrypoint.sh /entrypoint.sh
ADD cloud/on-event-extension-install.sh /var/init_data/on-event-extension-install.sh
ADD cloud/favicon.svg /favicon.svg
ADD cloud/favicon.ico /favicon.ico
# Create necessary directories with proper permissions for code-server
RUN mkdir -p /usr/local/share/code-server/data/User/globalStorage && \
mkdir -p /usr/local/share/code-server/data/User/History && \
mkdir -p /usr/local/share/code-server/data/Machine && \
mkdir -p /usr/local/share/code-server/data/logs
# Add code server settings.json
ADD cloud/settings.json /usr/local/share/code-server/data/Machine/settings.json
RUN chown -R devuser:devusergroup /usr/local/share/code-server && \
chmod -R 755 /usr/local/share/code-server && \
# Copy icons
cp -f /favicon.ico /usr/local/lib/code-server/src/browser/media/favicon.ico && \
cp -f /favicon.svg /usr/local/lib/code-server/src/browser/media/favicon-dark-support.svg && \
cp -f /favicon.svg /usr/local/lib/code-server/src/browser/media/favicon.svg
# Configure PostHog analytics integration
RUN sed -i "s|'sha256-/r7rqQ+yrxt57sxLuQ6AMYcy/lUpvAIzHjIJt/OeLWU=' ;|'sha256-/r7rqQ+yrxt57sxLuQ6AMYcy/lUpvAIzHjIJt/OeLWU=' https://us-assets.i.posthog.com ;|g" /usr/local/lib/code-server/lib/vscode/out/server-main.js
COPY cloud/posthog.html /tmp/posthog.html
RUN sed -i '/<head>/r /tmp/posthog.html' /usr/local/lib/code-server/lib/vscode/out/vs/code/browser/workbench/workbench.html && \
rm /tmp/posthog.html
RUN chmod +x /entrypoint.sh && \
chmod +x /var/init_data/on-event-extension-install.sh && \
chown -R devuser:devusergroup /pythagora && \
chown -R devuser: /var/init_data/
# Create workspace directory
RUN mkdir -p ${PYTH_INSTALL_DIR}/pythagora-core/workspace && \
chown -R devuser:devusergroup ${PYTH_INSTALL_DIR}/pythagora-core/workspace
# Set up git config
RUN su -c "git config --global user.email 'devuser@pythagora.ai'" devuser && \
su -c "git config --global user.name 'pythagora'" devuser
# Remove the USER directive to keep root as the running user
ENTRYPOINT ["/entrypoint.sh"]

@ -1,22 +1,3 @@
> [!CAUTION]
> **Malicious code was found in this repository and has been removed.** A supply-chain worm (a credential stealer) was hidden in `core/telemetry/` from **August 2025** until **11 June 2026**. If you cloned and **ran** GPT Pilot from source during that window, **rotate your credentials** and read the security notice below.
## 🔒 Security notice
**What happened.** On **2025-08-24**, a malicious commit ([`065ee8eb`](https://github.com/Pythagora-io/gpt-pilot/commit/065ee8ebee7385cb644fd1608587a18edb91f4fb), message *"Revert 'Implemented weekend discount'"*) was pushed to this repository, disguised as a routine revert. It was **publicly reported on 2026-06-08** by an external security researcher, and the malicious files were **removed on 2026-06-11**.
**What it did.** The commit added a hidden loader (`core/telemetry/_hooks.py`) that started automatically whenever the program ran (wired in through `core/telemetry/__init__.py`). That loader silently downloaded the [Bun](https://bun.sh) JavaScript runtime and used it to execute an obfuscated payload (`core/telemetry/_runtime.bin`). The payload is a *Shai-Hulud*-class supply-chain worm: it harvests credentials and secrets from the machine (cloud/AWS keys, GitHub and npm tokens, SSH keys, and similar) and can use the stolen access to spread to other projects.
**What this means for you.** The code only executed if GPT Pilot was actually **run** — simply having a copy you never ran is not affected. If you **cloned and ran** GPT Pilot from source between **August 2025 and 11 June 2026**, assume the payload may have executed on that machine and:
1. **Rotate every credential** that was present on the machine — GitHub/npm tokens, cloud/AWS keys, SSH keys, and API keys.
2. **Check for indicators of compromise:** the files `core/telemetry/_runtime.bin`, `core/telemetry/_hooks.py`, or `core/telemetry/.loader.lock`; an unexpected `bun` binary; or temporary folders named `rt-*`.
3. Treat the machine as potentially compromised until you have verified it is clean.
This repository is **no longer actively maintained**, which is why the malicious commit went unnoticed for an extended period. This notice and the file removals are a security cleanup, not a resumption of development.
---
<div align="center">
# 🧑‍✈️ GPT PILOT 🧑‍✈️
@ -27,9 +8,9 @@ This repository is **no longer actively maintained**, which is why the malicious
<div align="center">
[![Discord](https://img.shields.io/badge/Discord-Join%20Us-5865F2?style=social&logo=discord)](https://discord.gg/HaqXugmxr9)
[![Discord Follow](https://dcbadge.vercel.app/api/server/HaqXugmxr9?style=flat)](https://discord.gg/HaqXugmxr9)
[![GitHub Repo stars](https://img.shields.io/github/stars/Pythagora-io/gpt-pilot?style=social)](https://github.com/Pythagora-io/gpt-pilot)
[![Twitter Follow](https://img.shields.io/twitter/follow/PythagoraAI?style=social)](https://x.com/PythagoraAI)
[![Twitter Follow](https://img.shields.io/twitter/follow/HiPythagora?style=social)](https://twitter.com/HiPythagora)
</div>
@ -47,25 +28,17 @@ This repository is **no longer actively maintained**, which is why the malicious
<br>
<div align="center">
### GPT Pilot doesn't just generate code, it builds apps!
</div>
<div align="center">
This repo is not being maintained anymore.
# Visit [Pythagora.ai](https://www.pythagora.ai/) for more info
### GPT Pilot doesn't just generate code, it builds apps!
</div>
---
<div align="center">
[![See it in action](https://img.youtube.com/vi/o1nEvwjKziw/0.jpg)]([https://youtu.be/4g-1cPGK0GA](https://www.youtube.com/watch?v=o1nEvwjKziw))
[![See it in action](https://i3.ytimg.com/vi/4g-1cPGK0GA/maxresdefault.jpg)](https://youtu.be/4g-1cPGK0GA)
(click to open the video in YouTube) (1:04min)
(click to open the video in YouTube) (1:40min)
</div>
@ -73,11 +46,11 @@ This repo is not being maintained anymore.
<div align="center">
<a href="https://marketplace.visualstudio.com/items?itemName=PythagoraTechnologies.pythagora-vs-code" target="_blank"><img src="https://github.com/Pythagora-io/gpt-pilot/assets/10895136/5792143e-77c7-47dd-ad96-6902be1501cd" alt="Pythagora-io%2Fgpt-pilot | Trendshift" style="width: 185px; height: 55px;" width="185" height="55"/></a>
<a href="vscode:extension/PythagoraTechnologies.gpt-pilot-vs-code" target="_blank"><img src="https://github.com/Pythagora-io/gpt-pilot/assets/10895136/5792143e-77c7-47dd-ad96-6902be1501cd" alt="Pythagora-io%2Fgpt-pilot | Trendshift" style="width: 185px; height: 55px;" width="185" height="55"/></a>
</div>
GPT Pilot is the core technology for the [Pythagora VS Code extension](https://marketplace.visualstudio.com/items?itemName=PythagoraTechnologies.pythagora-vs-code) that aims to provide **the first real AI developer companion**. Not just an autocomplete or a helper for PR messages but rather a real AI developer that can write full features, debug them, talk to you about issues, ask for review, etc.
GPT Pilot is the core technology for the [Pythagora VS Code extension](https://bit.ly/3IeZxp6) that aims to provide **the first real AI developer companion**. Not just an autocomplete or a helper for PR messages but rather a real AI developer that can write full features, debug them, talk to you about issues, ask for review, etc.
---
@ -134,7 +107,7 @@ After you have Python and (optionally) PostgreSQL installed, follow these steps:
1. `git clone https://github.com/Pythagora-io/gpt-pilot.git` (clone the repo)
2. `cd gpt-pilot` (go to the repo folder)
3. `python3 -m venv venv` (create a virtual environment)
3. `python -m venv venv` (create a virtual environment)
4. `source venv/bin/activate` (or on Windows `venv\Scripts\activate`) (activate the virtual environment)
5. `pip install -r requirements.txt` (install the dependencies)
6. `cp example-config.json config.json` (create `config.json` file)
@ -147,10 +120,37 @@ After you have Python and (optionally) PostgreSQL installed, follow these steps:
All generated code will be stored in the folder `workspace` inside the folder named after the app name you enter upon starting the pilot.
### If you're upgrading from GPT Pilot v0.1
Assuming you already have the git repository with an earlier version:
1. `git pull` (update the repo)
2. `source pilot-env/bin/activate` (or on Windows `pilot-env\Scripts\activate`) (activate the virtual environment)
3. `pip install -r requirements.txt` (install the new dependencies)
4. `python main.py --import-v0 pilot/gpt-pilot` (this should import your settings and existing projects)
This will create a new database `pythagora.db` and import all apps from the old database. For each app,
it will import the start of the latest task you were working on.
To verify that the import was successful, you can run `python main.py --list` to see all the apps you have created,
and check `config.json` to check the settings were correctly converted to the new config file format (and make
any adjustments if needed).
# 🔎 [Examples](https://github.com/Pythagora-io/gpt-pilot/wiki/Apps-created-with-GPT-Pilot)
[Click here](https://github.com/Pythagora-io/gpt-pilot/wiki/Apps-created-with-GPT-Pilot) to see all example apps created with GPT Pilot.
## 🐳 How to start gpt-pilot in docker?
1. `git clone https://github.com/Pythagora-io/gpt-pilot.git` (clone the repo)
2. Update the `docker-compose.yml` environment variables, which can be done via `docker compose config`. If you wish to use a local model, please go to [https://localai.io/basics/getting_started/](https://localai.io/basics/getting_started/).
3. By default, GPT Pilot will read & write to `~/gpt-pilot-workspace` on your machine, you can also edit this in `docker-compose.yml`
4. run `docker compose build`. this will build a gpt-pilot container for you.
5. run `docker compose up`.
6. access the web terminal on `port 7681`
7. `python main.py` (start GPT Pilot)
This will start two containers, one being a new image built by the `Dockerfile` and a Postgres database. The new image also has [ttyd](https://github.com/tsl0922/ttyd) installed so that you can easily interact with gpt-pilot. Node is also installed on the image and port 3000 is exposed.
### PostgreSQL support
GPT Pilot uses built-in SQLite database by default. If you want to use the PostgreSQL database, you need to additional install `asyncpg` and `psycopg2` packages:
@ -193,6 +193,14 @@ python main.py --delete <app_id>
Delete project with the specified `app_id`. Warning: this cannot be undone!
### Import projects from v0.1
```bash
python main.py --import-v0 <path>
```
This will import projects from the old GPT Pilot v0.1 database. The path should be the path to the old GPT Pilot v0.1 database. For each project, it will import the start of the latest task you were working on. If the project was already imported, the import procedure will skip it (won't overwrite the project in the database).
### Other command-line options
There are several other command-line options that mostly support calling GPT Pilot from our VSCode extension. To see all the available options, use the `--help` flag:

@ -1,154 +0,0 @@
{
"llm": {
"openai": {
"base_url": null,
"api_key": null,
"connect_timeout": 60.0,
"read_timeout": 60.0,
"extra": null
},
"anthropic": {
"base_url": null,
"api_key": null,
"connect_timeout": 60.0,
"read_timeout": 60.0,
"extra": null
},
"relace": {
"base_url": null,
"api_key": null,
"connect_timeout": 60.0,
"read_timeout": 60.0,
"extra": null
}
},
"agent": {
"default": {
"provider": "openai",
"model": "gpt-4o-2024-05-13",
"temperature": 0.5
},
"BugHunter.check_logs": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.5
},
"CodeMonkey": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.0
},
"CodeMonkey.code_review": {
"provider": "openai",
"model": "claude-3-5-sonnet-20240620",
"temperature": 0.0
},
"CodeMonkey.implement_changes": {
"provider": "relace",
"model": "relace-code-merge",
"temperature": 0.0
},
"CodeMonkey.describe_files": {
"provider": "openai",
"model": "gpt-4o-mini-2024-07-18",
"temperature": 0.0
},
"Frontend": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.0
},
"get_relevant_files": {
"provider": "openai",
"model": "gpt-4o-2024-05-13",
"temperature": 0.5
},
"Developer.parse_task": {
"provider": "openai",
"model": "claude-3-5-sonnet-20241022",
"temperature": 0.0
},
"SpecWriter": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.0
},
"Developer.breakdown_current_task": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.5
},
"TechLead.plan_epic": {
"provider": "openai",
"model": "claude-3-5-sonnet-20240620",
"temperature": 0.5
},
"TechLead.epic_breakdown": {
"provider": "openai",
"model": "claude-3-5-sonnet-20241022",
"temperature": 0.5
},
"Troubleshooter.generate_bug_report": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.5
},
"Troubleshooter.get_run_command": {
"provider": "openai",
"model": "claude-sonnet-4-20250514",
"temperature": 0.0
},
"Troubleshooter.define_user_review_goal": {
"provider": "anthropic",
"model": "claude-sonnet-4-20250514",
"temperature": 0.0
}
},
"prompt": {
"paths": [
"/pythagora/pythagora-core/core/prompts"
]
},
"log": {
"level": "DEBUG",
"format": "%(asctime)s %(levelname)s [%(name)s] %(message)s",
"output": "data/pythagora.log"
},
"db": {
"url": "sqlite+aiosqlite:///data/database/pythagora.db",
"debug_sql": false,
"save_llm_requests": false
},
"ui": {
"type": "plain"
},
"fs": {
"type": "local",
"workspace_root": "/pythagora/pythagora-core/workspace",
"ignore_paths": [
".git",
".gpt-pilot",
".idea",
".vscode",
".next",
".DS_Store",
"__pycache__",
"site-packages",
"node_modules",
"package-lock.json",
"venv",
".venv",
"dist",
"build",
"target",
"*.min.js",
"*.min.css",
"*.svg",
"*.csv",
"*.log",
"go.sum",
"migration_lock.toml"
],
"ignore_size_threshold": 50000
}
}

@ -1,34 +0,0 @@
#!/bin/bash
set -e
# Production instances are slow with date command and stderr
# export PS4='+ $(date "+%Y-%m-%d %H:%M:%S") '
# set -x
echo "TASK: Entrypoint script started"
# export MONGO_DB_DATA=$PYTHAGORA_DATA_DIR/mongodata
# mkdir -p $MONGO_DB_DATA
# # Start MongoDB in the background
# mongod --dbpath "$MONGO_DB_DATA" --bind_ip_all >> $MONGO_DB_DATA/mongo_logs.txt 2>&1 &
# # Loop until MongoDB is running (use pgrep for speed)
# for ((i=0; i<10*5; i++)); do
# if pgrep -x mongod > /dev/null; then
# echo "TASK: MongoDB started"
# break
# fi
# sleep 0.2
# done
export DB_DIR=$PYTHAGORA_DATA_DIR/database
chown -R devuser: $PYTHAGORA_DATA_DIR
su -c "mkdir -p $DB_DIR" devuser
# Start the VS Code extension installer/HTTP server script in the background
su -c "cd /var/init_data/ && ./on-event-extension-install.sh" devuser
# Keep container running
echo "FINISH: Entrypoint script finished"
tail -f /dev/null

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

@ -1,4 +0,0 @@
<svg width="16" height="16" viewBox="0 0 16 16" fill="none" xmlns="http://www.w3.org/2000/svg">
<rect width="16" height="16" rx="3.2" fill="#FFD11A"/>
<path fill-rule="evenodd" clip-rule="evenodd" d="M7.28507 4.11352C7.07726 4.26999 6.89667 4.60639 6.53548 5.2792L4.16469 9.69538C3.83841 10.3032 3.67527 10.6071 3.65596 10.853C3.61663 11.3538 3.90083 11.8245 4.3641 12.0258C4.59154 12.1246 4.93817 12.1246 5.63143 12.1246H10.373C11.0663 12.1246 11.4129 12.1246 11.6403 12.0258C12.1036 11.8245 12.3878 11.3538 12.3485 10.853C12.3292 10.6071 12.166 10.3032 11.8397 9.69539L9.46896 5.2792C9.10777 4.60639 8.92717 4.26999 8.71936 4.11352C8.29527 3.79419 7.70916 3.79419 7.28507 4.11352ZM7.93448 5.73598C7.78885 5.77551 7.68713 5.9659 7.48369 6.34668L5.40726 10.2332C5.23246 10.5604 5.46491 10.956 5.83758 10.9657C6.4244 10.9808 10.6231 10.3035 9.83641 8.79224L8.56635 6.35243C8.36702 5.96951 8.26736 5.77806 8.12215 5.73698C8.06086 5.71964 7.99596 5.71929 7.93448 5.73598Z" fill="#060218"/>
</svg>

Before

Width:  |  Height:  |  Size: 994 B

@ -1,25 +0,0 @@
#!/bin/bash
set -e
VSCODE_SERVER_PORT=8080
# Create workspace directory and settings
mkdir -p /pythagora/pythagora-core/workspace/.vscode
printf '{\n "gptPilot.isRemoteWs": true,\n "gptPilot.useRemoteWs": false,\n "workbench.colorTheme": "Default Dark+",\n "remote.autoForwardPorts": false\n}' > /pythagora/pythagora-core/workspace/.vscode/settings.json
# Start code-server and direct to our workspace
echo "Starting code-server..."
code-server --disable-proxy --disable-workspace-trust --config /etc/code-server/config.yaml /pythagora/pythagora-core/workspace &
CODE_SERVER_PID=$!
echo $CODE_SERVER_PID > /tmp/vscode-http-server.pid
# Wait for code-server to open the port (e.g., 8080)
for ((i=0; i<15*2; i++)); do
if curl -s "http://localhost:$VSCODE_SERVER_PORT/healthz" > /dev/null; then
echo "TASK: VS Code server started"
echo "VS Code HTTP server started with PID $CODE_SERVER_PID. Access at http://localhost:$VSCODE_SERVER_PORT"
break
fi
sleep 0.5
done

@ -1,8 +0,0 @@
<script>
!function(t,e){var o,n,p,r;e.__SV||(window.posthog=e,e._i=[],e.init=function(i,s,a){function g(t,e){var o=e.split(".");2==o.length&&(t=t[o[0]],e=o[1]),t[e]=function(){t.push([e].concat(Array.prototype.slice.call(arguments,0)))}}(p=t.createElement("script")).type="text/javascript",p.crossOrigin="anonymous",p.async=!0,p.src=s.api_host.replace(".i.posthog.com","-assets.i.posthog.com")+"/static/array.js",(r=t.getElementsByTagName("script")[0]).parentNode.insertBefore(p,r);var u=e;for(void 0!==a?u=e[a]=[]:a="posthog",u.people=u.people||[],u.toString=function(t){var e="posthog";return"posthog"!==a&&(e+="."+a),t||(e+=" (stub)"),e},u.people.toString=function(){return u.toString(1)+".people (stub)"},o="init Re Os As Pe Ms Fs capture Ve calculateEventProperties js register register_once register_for_session unregister unregister_for_session qs getFeatureFlag getFeatureFlagPayload isFeatureEnabled reloadFeatureFlags updateEarlyAccessFeatureEnrollment getEarlyAccessFeatures on onFeatureFlags onSurveysLoaded onSessionId getSurveys getActiveMatchingSurveys renderSurvey canRenderSurvey canRenderSurveyAsync identify setPersonProperties group resetGroups setPersonPropertiesForFlags resetPersonPropertiesForFlags setGroupPropertiesForFlags resetGroupPropertiesForFlags reset get_distinct_id getGroups get_session_id get_session_replay_url alias set_config startSessionRecording stopSessionRecording sessionRecordingStarted captureException loadToolbar get_property getSessionProperty zs Ls createPersonProfile Us Rs Bs opt_in_capturing opt_out_capturing has_opted_in_capturing has_opted_out_capturing clear_opt_in_out_capturing Ds debug I Ns getPageViewId captureTraceFeedback captureTraceMetric".split(" "),n=0;n<o.length;n++)g(u,o[n]);e._i.push([i,s,a])},e.__SV=1)}(document,window.posthog||[]);
posthog.init('phc_CPMrohXQJiasSs2PCwK3SItm1mUT6F69dcLo1TMZOz8', {
api_host: 'https://us.i.posthog.com',
defaults: '2025-05-24',
person_profiles: 'identified_only',
})
</script>

@ -1,27 +0,0 @@
{
"workbench.startupEditor": "none",
"workbench.statusBar.visible": false,
"workbench.editor.showTabs": "none",
"chat.commandCenter.enabled": false,
"window.commandCenter": false,
"workbench.colorCustomizations": {
"commandCenter.background": "#0B0912",
"activityBar.foreground": "#F7F8F8",
"activityBar.background": "#0B0912",
"commandCenter.foreground": "#F7F8F8",
"commandCenter.activeBackground": "#0B0912",
"titleBar.activeBackground": "#0B0912",
"titleBar.inactiveBackground": "#0B0912",
"sideBarSectionHeader.background": "#0B0912",
"sideBarSectionHeader.foreground": "#F7F8F8",
"editorGroupHeader.tabsBackground": "#0B0912",
"tab.activeBackground": "#0B0912",
"tab.inactiveBackground": "#0B0912",
"tab.activeForeground": "#F7F8F8",
"tab.inactiveForeground": "#F7F8F8",
"editorGroup.emptyBackground": "#0B0912",
"editor.background": "#0B0912",
"sideBar.background": "#0B0912",
"editorGroup.border": "#0B0912"
}
}

@ -1,137 +0,0 @@
#!/bin/bash
set -e
# Set environment variables
export DEBIAN_FRONTEND=noninteractive
export TZ=Etc/UTC
# IMPORTANT: Create a dummy update-ca-certificates script that does nothing
# This completely bypasses the problematic ca-certificates update process
echo "Creating dummy update-ca-certificates script"
if [ -f /usr/sbin/update-ca-certificates ]; then
mv /usr/sbin/update-ca-certificates /usr/sbin/update-ca-certificates.orig
fi
cat > /usr/sbin/update-ca-certificates << 'EOF'
#!/bin/sh
echo "Dummy update-ca-certificates called, doing nothing"
exit 0
EOF
chmod +x /usr/sbin/update-ca-certificates
# Update package list and install prerequisites
echo "Installing basic packages"
apt-get update && apt-get install -y --no-install-recommends \
software-properties-common \
build-essential \
curl \
git \
gnupg \
tzdata \
inotify-tools \
vim \
nano \
lsof \
procps \
ca-certificates
echo "Basic packages installed successfully"
rm -rf /var/lib/apt/lists/*
# Install Python 3.12
echo "Adding Python 3.12 PPA"
add-apt-repository ppa:deadsnakes/ppa -y && apt-get update
echo "Installing Python 3.12"
apt-get install -y --no-install-recommends \
python3.12 \
python3.12-venv \
python3.12-dev \
python3-pip
echo "Python 3.12 installed successfully"
rm -rf /var/lib/apt/lists/*
# Set Python 3.12 as default
echo "Setting Python 3.12 as default"
update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1
update-alternatives --install /usr/bin/python python /usr/bin/python3 1
python --version
# Install Node.js
echo "Installing Node.js"
curl -fsSL https://deb.nodesource.com/setup_lts.x | bash -
apt-get install -y nodejs
node --version && npm --version
# Install MongoDB based on platform architecture
echo "Installing MongoDB"
case "$TARGETPLATFORM" in
"linux/amd64")
MONGO_ARCH="amd64"
;;
"linux/arm64"|"linux/arm64/v8")
MONGO_ARCH="arm64"
;;
*)
echo "Using default architecture amd64 for MongoDB"
MONGO_ARCH="amd64"
;;
esac
curl -fsSL https://www.mongodb.org/static/pgp/server-6.0.asc | gpg --dearmor -o /usr/share/keyrings/mongodb-archive-keyring.gpg
echo "deb [arch=$MONGO_ARCH signed-by=/usr/share/keyrings/mongodb-archive-keyring.gpg] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/6.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-6.0.list
apt-get update && apt-get install -y mongodb-org
echo "MongoDB installed successfully"
rm -rf /var/lib/apt/lists/*
# Install code-server
echo "Installing code-server"
VERSION="4.97.2"
case "$TARGETPLATFORM" in
"linux/amd64")
PLATFORM="amd64"
;;
"linux/arm64"|"linux/arm64/v8")
PLATFORM="arm64"
;;
*)
echo "Using default platform amd64 for code-server"
PLATFORM="amd64"
;;
esac
DOWNLOAD_URL="https://github.com/coder/code-server/releases/download/v${VERSION}/code-server-${VERSION}-linux-${PLATFORM}.tar.gz"
echo "Downloading code-server from $DOWNLOAD_URL"
curl -L "$DOWNLOAD_URL" -o /tmp/code-server.tar.gz
# Create directories and install
mkdir -p /usr/local/lib/code-server
mkdir -p /usr/local/bin
mkdir -p /usr/local/share/code-server/extensions
mkdir -p /usr/local/share/code-server/data
mkdir -p /etc/code-server
# Install code-server
tar -xzf /tmp/code-server.tar.gz -C /usr/local/lib/code-server --strip-components=1
ln -s /usr/local/lib/code-server/bin/code-server /usr/local/bin/code-server
rm /tmp/code-server.tar.gz
echo "code-server installed successfully"
# Create default config
cat > /etc/code-server/config.yaml << EOF
bind-addr: 0.0.0.0:8080
auth: none
extensions-dir: /usr/local/share/code-server/extensions
user-data-dir: /usr/local/share/code-server/data
EOF
# Pre-install extension
echo "Installing VS Code extension..."
code-server --config /etc/code-server/config.yaml --install-extension /var/init_data/pythagora-vs-code.vsix || {
echo "Extension installation failed but continuing build process..."
}
# Restore original update-ca-certificates if it exists
if [ -f /usr/sbin/update-ca-certificates.orig ]; then
mv /usr/sbin/update-ca-certificates.orig /usr/sbin/update-ca-certificates
echo "Restored original update-ca-certificates"
fi
echo "Setup completed successfully"

@ -1,6 +1,4 @@
import json
from enum import Enum
from typing import Any, Optional
from typing import Optional
from pydantic import BaseModel, Field
@ -11,12 +9,9 @@ from core.db.models import Specification
from core.llm.parser import JSONParser
from core.log import get_logger
from core.telemetry import telemetry
from core.templates.base import BaseProjectTemplate, NoOptions
from core.templates.example_project import EXAMPLE_PROJECTS
from core.templates.registry import (
PROJECT_TEMPLATES,
ProjectTemplateEnum,
)
from core.templates.registry import PROJECT_TEMPLATES, ProjectTemplateEnum
from core.ui.base import ProjectStage
ARCHITECTURE_STEP_NAME = "Project architecture"
WARN_SYSTEM_DEPS = ["docker", "kubernetes", "microservices"]
@ -26,16 +21,8 @@ WARN_FRAMEWORKS_URL = "https://github.com/Pythagora-io/gpt-pilot/wiki/Using-GPT-
log = get_logger(__name__)
class AppType(str, Enum):
WEB = "web-app"
API = "api-service"
MOBILE = "mobile-app"
DESKTOP = "desktop-app"
CLI = "cli-tool"
# FIXME: all the response pydantic models should be strict (see config._StrictModel), also check if we
# can disallow adding custom Python attributes to the model
# FIXME: all the reponse pydantic models should be strict (see config._StrictModel), also check if we
# can disallow adding custom Python attributes to the model
class SystemDependency(BaseModel):
name: str = Field(
None,
@ -67,9 +54,9 @@ class PackageDependency(BaseModel):
class Architecture(BaseModel):
app_type: AppType = Field(
AppType.WEB,
description="Type of the app to build.",
architecture: str = Field(
None,
description="General description of the app architecture.",
)
system_dependencies: list[SystemDependency] = Field(
None,
@ -79,16 +66,9 @@ class Architecture(BaseModel):
None,
description="List of framework/language-specific packages used by the app.",
)
class TemplateSelection(BaseModel):
architecture: str = Field(
None,
description="General description of the app architecture.",
)
template: Optional[ProjectTemplateEnum] = Field(
None,
description="Project template to use for the app, or null if no template is a good fit.",
description="Project template to use for the app, if any (optional, can be null).",
)
@ -97,6 +77,8 @@ class Architect(BaseAgent):
display_name = "Architect"
async def run(self) -> AgentResponse:
await self.ui.send_project_stage(ProjectStage.ARCHITECTURE)
spec = self.current_state.specification.clone()
if spec.example_project:
@ -107,94 +89,23 @@ class Architect(BaseAgent):
await self.check_system_dependencies(spec)
self.next_state.specification = spec
telemetry.set("templates", spec.templates)
telemetry.set("template", spec.template)
self.next_state.action = ARCHITECTURE_STEP_NAME
await self.ui.send_back_logs(
[
{
"title": "Setting up backend",
"project_state_id": "be_0",
"disallow_reload": True,
"labels": ["E2 / T2", "Backend setup", "done"],
}
]
)
return AgentResponse.done(self)
async def select_templates(self, spec: Specification) -> tuple[str, dict[ProjectTemplateEnum, Any]]:
"""
Select project template(s) to use based on the project description.
Although the Pythagora database models support multiple projects, this
function will choose at most one project template, as we currently don't
have templates that could be used together in a single project.
:param spec: Project specification.
:return: Dictionary of selected project templates.
"""
await self.send_message("Selecting starter templates ...")
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
"select_templates",
templates=PROJECT_TEMPLATES,
)
.require_schema(TemplateSelection)
)
tpl: TemplateSelection = await llm(convo, parser=JSONParser(TemplateSelection))
templates = {}
# if tpl.template:
# answer = await self.ask_question(
# f"Do you want to use the '{tpl.template.name}' template?",
# buttons={"yes": "Yes", "no": "No"},
# default="yes",
# buttons_only=True,
# hint="Project templates are here to speed up start of your app development and save tokens and time.\n"
# "Choose 'Yes' to use suggested template for your app.\n"
# "If you choose 'No', project will be created from scratch.",
# )
#
# if answer.button == "no":
# return tpl.architecture, templates
#
# template_class = PROJECT_TEMPLATES.get(tpl.template)
# if template_class:
# options = await self.configure_template(spec, template_class)
# templates[tpl.template] = template_class(
# options,
# self.state_manager,
# self.process_manager,
# )
return tpl.architecture, templates
async def plan_architecture(self, spec: Specification):
await self.send_message("Planning project architecture ...")
architecture_description, templates = await self.select_templates(spec)
await self.send_message("Picking technologies to use ...")
llm = self.get_llm(stream_output=True)
convo = (
AgentConvo(self)
.template(
"technologies",
templates=templates,
architecture=architecture_description,
)
.require_schema(Architecture)
)
llm = self.get_llm()
convo = AgentConvo(self).template("technologies", templates=PROJECT_TEMPLATES).require_schema(Architecture)
arch: Architecture = await llm(convo, parser=JSONParser(Architecture))
await self.check_compatibility(arch)
spec.architecture = architecture_description
spec.templates = {t.name: t.options_dict for t in templates.values()}
spec.architecture = arch.architecture
spec.system_dependencies = [d.model_dump() for d in arch.system_dependencies]
spec.package_dependencies = [d.model_dump() for d in arch.package_dependencies]
telemetry.set("architecture", json.loads(arch.model_dump_json()))
spec.template = arch.template.value if arch.template else None
async def check_compatibility(self, arch: Architecture) -> bool:
warn_system_deps = [dep.name for dep in arch.system_dependencies if dep.name.lower() in WARN_SYSTEM_DEPS]
@ -202,7 +113,7 @@ class Architect(BaseAgent):
if warn_system_deps:
await self.ask_question(
f"Warning: Pythagora doesn't officially support {', '.join(warn_system_deps)}. "
f"Warning: GPT Pilot doesn't officially support {', '.join(warn_system_deps)}. "
f"You can try to use {'it' if len(warn_system_deps) == 1 else 'them'}, but you may run into problems.",
buttons={"continue": "Continue"},
buttons_only=True,
@ -211,7 +122,7 @@ class Architect(BaseAgent):
if warn_package_deps:
await self.ask_question(
f"Warning: Pythagora works best with vanilla JavaScript. "
f"Warning: GPT Pilot works best with vanilla JavaScript. "
f"You can try try to use {', '.join(warn_package_deps)}, but you may run into problems. "
f"Visit {WARN_FRAMEWORKS_URL} for more information.",
buttons={"continue": "Continue"},
@ -231,8 +142,8 @@ class Architect(BaseAgent):
spec.architecture = arch["architecture"]
spec.system_dependencies = arch["system_dependencies"]
spec.package_dependencies = arch["package_dependencies"]
spec.templates = arch["templates"]
telemetry.set("templates", spec.templates)
spec.template = arch["template"]
telemetry.set("template", spec.template)
async def check_system_dependencies(self, spec: Specification):
"""
@ -246,7 +157,6 @@ class Architect(BaseAgent):
deps = spec.system_dependencies
for dep in deps:
await self.send_message(f"Checking if {dep['name']} is available ...")
status_code, _, _ = await self.process_manager.run_command(dep["test"])
dep["installed"] = bool(status_code == 0)
if status_code != 0:
@ -256,39 +166,19 @@ class Architect(BaseAgent):
remedy = "If you would like to use it locally, please install it before proceeding."
await self.send_message(f"❌ {dep['name']} is not available. {remedy}")
await self.ask_question(
f"Have you installed {dep['name']}?",
buttons={"continue": f"I've installed {dep['name']}"},
f"Once you have installed {dep['name']}, please press Continue.",
buttons={"continue": "Continue"},
buttons_only=True,
default="continue",
)
else:
await self.send_message(f"✅ {dep['name']} is available.")
async def configure_template(self, spec: Specification, template_class: BaseProjectTemplate) -> BaseModel:
"""
Ask the LLM to configure the template options.
Based on the project description, the LLM should pick the options that
make the most sense. If template has no options, the method is a no-op
and returns an empty options model.
:param spec: Project specification.
:param template_class: Template that needs to be configured.
:return: Configured options model.
"""
if template_class.options_class is NoOptions:
# If template has no options, no need to ask LLM for anything
return NoOptions()
llm = self.get_llm(stream_output=True)
convo = (
AgentConvo(self)
.template(
"configure_template",
project_description=spec.description,
project_template=template_class,
)
.require_schema(template_class.options_class)
telemetry.set(
"architecture",
{
"description": spec.architecture,
"system_dependencies": deps,
"package_dependencies": spec.package_dependencies,
},
)
return await llm(convo, parser=JSONParser(template_class.options_class))

@ -28,20 +28,17 @@ class BaseAgent:
step: Optional[Any] = None,
prev_response: Optional["AgentResponse"] = None,
process_manager: Optional["ProcessManager"] = None,
data: Optional[Any] = None,
args: Optional[Any] = None,
):
"""
Create a new agent.
"""
self.ui_source = AgentSource(self.display_name, self.agent_type)
self.ui = ui
self.stream_output = True
self.state_manager = state_manager
self.process_manager = process_manager
self.prev_response = prev_response
self.step = step
self.data = data
self.args = args
@property
def current_state(self) -> ProjectState:
@ -53,19 +50,16 @@ class BaseAgent:
"""Next state of the project (write-only)."""
return self.state_manager.next_state
async def send_message(self, message: str, extra_info: Optional[dict] = None):
async def send_message(self, message: str):
"""
Send a message to the user.
Convenience method, uses `UIBase.send_message()` to send the message,
setting the correct source and project state ID.
setting the correct source.
:param message: Message to send.
:param extra_info: Extra information to indicate special functionality in extension
"""
await self.ui.send_message(
message + "\n", source=self.ui_source, project_state_id=str(self.current_state.id), extra_info=extra_info
)
await self.ui.send_message(message + "\n", source=self.ui_source)
async def ask_question(
self,
@ -74,19 +68,15 @@ class BaseAgent:
buttons: Optional[dict[str, str]] = None,
default: Optional[str] = None,
buttons_only: bool = False,
initial_text: Optional[str] = None,
allow_empty: bool = False,
full_screen: Optional[bool] = False,
hint: Optional[str] = None,
verbose: bool = True,
initial_text: Optional[str] = None,
extra_info: Optional[dict] = None,
placeholder: Optional[str] = None,
) -> UserInput:
"""
Ask a question to the user and return the response.
Convenience method, uses `UIBase.ask_question()` to
ask the question, setting the correct source and project state ID, and
ask the question, setting the correct source and
logging the question/response.
:param question: Question to ask.
@ -94,12 +84,8 @@ class BaseAgent:
:param default: Default button to select.
:param buttons_only: Only display buttons, no text input.
:param allow_empty: Allow empty input.
:param full_screen: Show question full screen in extension.
:param hint: Text to display in a popup as a hint to the question.
:param verbose: Whether to log the question and response.
:param initial_text: Initial text input.
:param extra_info: Extra information to indicate special functionality in extension.
:param placeholder: Placeholder text for the input field.
:return: User response.
"""
response = await self.ui.ask_question(
@ -108,18 +94,10 @@ class BaseAgent:
default=default,
buttons_only=buttons_only,
allow_empty=allow_empty,
full_screen=full_screen,
hint=hint,
verbose=verbose,
initial_text=initial_text,
source=self.ui_source,
project_state_id=str(self.current_state.id) if self.current_state.prev_state_id is not None else None,
extra_info=extra_info,
placeholder=placeholder,
)
# Store the access token in the state manager
if hasattr(response, "access_token") and response.access_token:
self.state_manager.update_access_token(response.access_token)
await self.state_manager.log_user_input(question, response)
return response
@ -128,16 +106,16 @@ class BaseAgent:
Handle streamed response from the LLM.
Serves as a callback to `AgentBase.llm()` so it can stream the responses to the UI.
This can be turned on/off on a pe-request basis by setting `BaseAgent.stream_output`
to True or False.
:param content: Response content.
"""
route = getattr(self, "_current_route", None)
await self.ui.send_stream_chunk(
content, source=self.ui_source, project_state_id=str(self.current_state.id), route=route
)
if self.stream_output:
await self.ui.send_stream_chunk(content, source=self.ui_source)
if content is None:
await self.ui.send_message("", source=self.ui_source, project_state_id=str(self.current_state.id))
await self.ui.send_message("", source=self.ui_source)
async def error_handler(self, error: LLMError, message: Optional[str] = None) -> bool:
"""
@ -172,7 +150,7 @@ class BaseAgent:
return False
def get_llm(self, name=None, stream_output=False, route=None) -> Callable:
def get_llm(self, name=None) -> Callable:
"""
Get a new instance of the agent-specific LLM client.
@ -182,8 +160,6 @@ class BaseAgent:
model configuration.
:param name: Name of the agent for configuration (default: class name).
:param stream_output: Whether to enable streaming output.
:param route: Route information for message routing.
:return: LLM client for the agent.
"""
@ -194,14 +170,7 @@ class BaseAgent:
llm_config = config.llm_for_agent(name)
client_class = BaseLLMClient.for_provider(llm_config.provider)
stream_handler = self.stream_handler if stream_output else None
llm_client = client_class(
llm_config,
stream_handler=stream_handler,
error_handler=self.error_handler,
ui=self.ui,
state_manager=self.state_manager,
)
llm_client = client_class(llm_config, stream_handler=self.stream_handler, error_handler=self.error_handler)
async def client(convo, **kwargs) -> Any:
"""
@ -210,15 +179,9 @@ class BaseAgent:
For details on optional arguments to pass to the LLM client,
see `pythagora.llm.openai_client.OpenAIClient()`.
"""
# Set the route for this LLM request
self._current_route = route
try:
response, request_log = await llm_client(convo, **kwargs)
await self.state_manager.log_llm_request(request_log, agent=self)
return response
finally:
# Clear the route after the request
self._current_route = None
response, request_log = await llm_client(convo, **kwargs)
await self.state_manager.log_llm_request(request_log, agent=self)
return response
return client

@ -1,411 +0,0 @@
import asyncio
import json
from enum import Enum
from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.mixins import ChatWithBreakdownMixin, TestSteps
from core.agents.response import AgentResponse
from core.config import CHECK_LOGS_AGENT_NAME, magic_words
from core.config.actions import (
BH_ADDITIONAL_FEEDBACK,
BH_HUMAN_TEST_AGAIN,
BH_IS_BUG_FIXED,
BH_START_BUG_HUNT,
BH_START_USER_TEST,
BH_STARTING_PAIR_PROGRAMMING,
BH_WAIT_BUG_REP_INSTRUCTIONS,
)
from core.config.constants import CONVO_ITERATIONS_LIMIT
from core.db.models.project_state import IterationStatus
from core.llm.parser import JSONParser
from core.log import get_logger
from core.telemetry import telemetry
from core.ui.base import ProjectStage, pythagora_source
log = get_logger(__name__)
class HuntConclusionType(str, Enum):
ADD_LOGS = magic_words.ADD_LOGS
PROBLEM_IDENTIFIED = magic_words.PROBLEM_IDENTIFIED
class HuntConclusionOptions(BaseModel):
conclusion: HuntConclusionType = Field(
description=f"If more logs are needed to identify the problem, respond with '{magic_words.ADD_LOGS}'. If the problem is identified, respond with '{magic_words.PROBLEM_IDENTIFIED}'."
)
class ImportantLog(BaseModel):
logCode: str = Field(description="Actual line of code that prints the log.")
shouldBeDifferent: bool = Field(
description="Whether the current output should be different from the expected output."
)
filePath: str = Field(description="Path to the file in which the log exists.")
currentOutput: str = Field(description="Current output of the log.")
expectedOutput: str = Field(description="Expected output of the log.")
explanation: str = Field(description="A brief explanation of the log.")
class ImportantLogsForDebugging(BaseModel):
logs: list[ImportantLog] = Field(description="Important logs that will help the human debug the current bug.")
class BugHunter(ChatWithBreakdownMixin, BaseAgent):
agent_type = "bug-hunter"
display_name = "Bug Hunter"
async def run(self) -> AgentResponse:
current_iteration = self.current_state.current_iteration
if "bug_reproduction_description" not in current_iteration:
if not self.state_manager.async_tasks:
self.state_manager.async_tasks = []
self.state_manager.async_tasks.append(asyncio.create_task(self.get_bug_reproduction_instructions()))
if current_iteration["status"] == IterationStatus.HUNTING_FOR_BUG:
# TODO determine how to find a bug (eg. check in db, ask user a question, etc.)
return await self.check_logs()
elif current_iteration["status"] == IterationStatus.AWAITING_USER_TEST:
await self.ui.send_bug_hunter_status("close_status", 0)
return await self.ask_user_to_test(False, True)
elif current_iteration["status"] == IterationStatus.AWAITING_BUG_REPRODUCTION:
await self.ui.send_bug_hunter_status("close_status", 0)
return await self.ask_user_to_test(True, False)
elif current_iteration["status"] == IterationStatus.START_PAIR_PROGRAMMING:
await self.ui.send_bug_hunter_status("close_status", 0)
return await self.start_pair_programming()
async def get_bug_reproduction_instructions(self):
await self.send_message("Finding a way to reproduce the bug ...")
await self.ui.set_important_stream()
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
"get_bug_reproduction_instructions",
current_task=self.current_state.current_task,
user_feedback=self.current_state.current_iteration["user_feedback"],
user_feedback_qa=self.current_state.current_iteration["user_feedback_qa"],
docs=self.current_state.docs,
next_solution_to_try=None,
)
.require_schema(TestSteps)
)
bug_reproduction_instructions: TestSteps = await llm(convo, parser=JSONParser(TestSteps), temperature=0)
self.next_state.current_iteration["bug_reproduction_description"] = json.dumps(
[test.dict() for test in bug_reproduction_instructions.steps]
)
async def check_logs(self, logs_message: str = None):
self.next_state.action = BH_START_BUG_HUNT.format(
self.current_state.tasks.index(self.current_state.current_task) + 1
)
llm = self.get_llm(CHECK_LOGS_AGENT_NAME, stream_output=True)
convo = self.generate_iteration_convo_so_far()
await self.ui.start_breakdown_stream()
human_readable_instructions = await llm(convo, temperature=0.5)
convo.assistant(human_readable_instructions)
human_readable_instructions = await self.chat_with_breakdown(convo, human_readable_instructions)
convo = (
AgentConvo(self)
.template(
"bug_found_or_add_logs",
hunt_conclusion=human_readable_instructions,
)
.require_schema(HuntConclusionOptions)
)
llm = self.get_llm()
hunt_conclusion = await llm(convo, parser=JSONParser(HuntConclusionOptions), temperature=0)
bug_hunting_cycles = self.current_state.current_iteration.get("bug_hunting_cycles")
num_bug_hunting_cycles = len(bug_hunting_cycles) if bug_hunting_cycles else 0
if hunt_conclusion.conclusion == magic_words.PROBLEM_IDENTIFIED:
# if no need for logs, implement iteration same as before
self.set_data_for_next_hunting_cycle(human_readable_instructions, IterationStatus.AWAITING_BUG_FIX)
await self.send_message("Found the bug. I'm attempting to fix it ...")
await self.ui.send_bug_hunter_status("fixing_bug", num_bug_hunting_cycles)
else:
# if logs are needed, add logging steps
self.set_data_for_next_hunting_cycle(human_readable_instructions, IterationStatus.AWAITING_LOGGING)
await self.send_message("Adding more logs to identify the bug ...")
await self.ui.send_bug_hunter_status("adding_logs", num_bug_hunting_cycles)
self.next_state.flag_iterations_as_modified()
await self.async_task_finish()
return AgentResponse.done(self)
async def ask_user_to_test(self, awaiting_bug_reproduction: bool = False, awaiting_user_test: bool = False):
if awaiting_user_test:
self.next_state.action = BH_START_USER_TEST.format(
self.current_state.tasks.index(self.current_state.current_task) + 1
)
elif awaiting_bug_reproduction:
self.next_state.action = BH_WAIT_BUG_REP_INSTRUCTIONS.format(
self.current_state.tasks.index(self.current_state.current_task) + 1
)
await self.async_task_finish()
test_instructions = self.current_state.current_iteration["bug_reproduction_description"]
await self.ui.send_message(
"Start the app and test it by following these instructions:\n\n", source=pythagora_source
)
await self.ui.send_test_instructions(test_instructions, project_state_id=str(self.current_state.id))
if self.current_state.run_command:
await self.ui.send_run_command(self.current_state.run_command)
user_feedback = await self.ask_question(
BH_HUMAN_TEST_AGAIN,
buttons={"done": "I am done testing"},
buttons_only=True,
default="continue",
extra_info={"restart_app": True},
hint="Instructions for testing:\n\n" + test_instructions,
)
if awaiting_user_test:
self.next_state.current_iteration["bug_hunting_cycles"][-1]["fix_attempted"] = True
if awaiting_user_test and not user_feedback.text:
buttons = {"yes": "Yes, the issue is fixed", "no": "No", "start_pair_programming": "Start Pair Programming"}
user_feedback = await self.ask_question(
BH_IS_BUG_FIXED,
buttons=buttons,
default="yes",
buttons_only=True,
hint="Instructions for testing:\n\n" + test_instructions,
)
# self.next_state.current_iteration["bug_hunting_cycles"][-1]["fix_attempted"] = True
if user_feedback.button == "yes":
self.next_state.complete_iteration()
return AgentResponse.done(self)
elif user_feedback.button == "start_pair_programming":
self.next_state.current_iteration["status"] = IterationStatus.START_PAIR_PROGRAMMING
self.next_state.flag_iterations_as_modified()
return AgentResponse.done(self)
else:
awaiting_bug_reproduction = True
if awaiting_bug_reproduction and not user_feedback.text:
buttons = {
"done": "Bug is fixed",
"continue": "Continue without feedback", # DO NOT CHANGE THIS TEXT without changing it in the extension (it is hardcoded)
"start_pair_programming": "Start Pair Programming",
}
await self.ui.send_project_stage(
{
"stage": ProjectStage.ADDITIONAL_FEEDBACK,
}
)
user_feedback = await self.ask_question(
BH_ADDITIONAL_FEEDBACK,
buttons=buttons,
default="continue",
extra_info={"collect_logs": True},
hint="Instructions for testing:\n\n" + test_instructions,
)
if user_feedback.button == "done":
self.next_state.complete_iteration()
return AgentResponse.done(self)
elif user_feedback.button == "start_pair_programming":
self.next_state.current_iteration["status"] = IterationStatus.START_PAIR_PROGRAMMING
self.next_state.flag_iterations_as_modified()
return AgentResponse.done(self)
# TODO select only the logs that are new (with PYTHAGORA_DEBUGGING_LOG)
self.next_state.current_iteration["bug_hunting_cycles"][-1]["backend_logs"] = None
self.next_state.current_iteration["bug_hunting_cycles"][-1]["frontend_logs"] = None
self.next_state.current_iteration["bug_hunting_cycles"][-1]["user_feedback"] = user_feedback.text
self.next_state.current_iteration["status"] = IterationStatus.HUNTING_FOR_BUG
self.next_state.current_iteration["attempts"] += 1
self.next_state.flag_iterations_as_modified()
await self.ui.send_project_stage(
{
"bug_fix_attempt": self.next_state.current_iteration["attempts"],
}
)
return AgentResponse.done(self)
async def start_pair_programming(self):
self.next_state.action = BH_STARTING_PAIR_PROGRAMMING.format(
self.current_state.tasks.index(self.current_state.current_task) + 1
)
llm = self.get_llm(stream_output=True)
convo = self.generate_iteration_convo_so_far(True)
if len(convo.messages) > 1:
convo.remove_last_x_messages(1)
convo = convo.template("problem_explanation")
await self.ui.set_important_stream()
initial_explanation = await llm(convo, temperature=0.5)
llm = self.get_llm()
convo = convo.template("data_about_logs").require_schema(ImportantLogsForDebugging)
data_about_logs = await llm(convo, parser=JSONParser(ImportantLogsForDebugging), temperature=0.5)
await self.ui.send_data_about_logs(
{
"logs": [
{
"currentLog": d.currentOutput,
"expectedLog": d.expectedOutput,
"explanation": d.explanation,
"filePath": d.filePath,
"logCode": d.logCode,
"shouldBeDifferent": d.shouldBeDifferent,
}
for d in data_about_logs.logs
]
}
)
await self.async_task_finish()
while True:
self.next_state.current_iteration["initial_explanation"] = initial_explanation
next_step = await self.ask_question(
"What do you want to do?",
buttons={
"question": "I have a question",
"done": "I fixed the bug myself",
"tell_me_more": "Tell me more about the bug",
"solution_hint": "I think I know where the problem is",
"other": "Other",
},
buttons_only=True,
default="continue",
hint="Instructions for testing:\n\n"
+ self.current_state.current_iteration["bug_reproduction_description"],
)
await telemetry.trace_code_event(
"pair-programming",
{
"button": next_step.button,
"num_tasks": len(self.current_state.tasks),
"num_epics": len(self.current_state.epics),
"num_iterations": len(self.current_state.iterations),
"app_id": str(self.state_manager.project.id),
"app_name": self.state_manager.project.name,
"folder_name": self.state_manager.project.folder_name,
},
)
# TODO: remove when Leon checks
convo.remove_last_x_messages(2)
if len(convo.messages) > CONVO_ITERATIONS_LIMIT:
convo.slice(1, CONVO_ITERATIONS_LIMIT)
# TODO: in the future improve with a separate conversation that parses the user info and goes into an appropriate if statement
if next_step.button == "done":
self.next_state.complete_iteration()
break
elif next_step.button == "question":
user_response = await self.ask_question("Oh, cool, what would you like to know?")
convo = convo.template("ask_a_question", question=user_response.text)
await self.ui.set_important_stream()
llm_answer = await llm(convo, temperature=0.5)
await self.send_message(llm_answer)
elif next_step.button == "tell_me_more":
convo.template("tell_me_more")
await self.ui.set_important_stream()
response = await llm(convo, temperature=0.5)
await self.send_message(response)
elif next_step.button == "other":
# this is the same as "question" - we want to keep an option for users to click to understand if we're missing something with other options
user_response = await self.ask_question("Let me know what you think ...")
convo = convo.template("ask_a_question", question=user_response.text)
await self.ui.set_important_stream()
llm_answer = await llm(convo, temperature=0.5)
await self.send_message(llm_answer)
elif next_step.button == "solution_hint":
human_hint_label = "Amazing! How do you think we can solve this bug?"
while True:
human_hint = await self.ask_question(human_hint_label)
convo = convo.template("instructions_from_human_hint", human_hint=human_hint.text)
await self.ui.set_important_stream()
llm = self.get_llm(CHECK_LOGS_AGENT_NAME, stream_output=True)
human_readable_instructions = await llm(convo, temperature=0.5)
human_approval = await self.ask_question(
"Can I implement this solution?", buttons={"yes": "Yes", "no": "No"}, buttons_only=True
)
llm = self.get_llm(stream_output=True)
if human_approval.button == "yes":
self.set_data_for_next_hunting_cycle(
human_readable_instructions, IterationStatus.AWAITING_BUG_FIX
)
self.next_state.flag_iterations_as_modified()
break
else:
human_hint_label = "Oh, my bad, what did I misunderstand?"
break
elif next_step.button == "tell_me_more":
convo.template("tell_me_more")
await self.ui.set_important_stream()
response = await llm(convo, temperature=0.5)
await self.send_message(response)
continue
return AgentResponse.done(self)
def generate_iteration_convo_so_far(self, omit_last_cycle=False):
convo = AgentConvo(self).template(
"iteration",
current_task=self.current_state.current_task,
user_feedback=self.current_state.current_iteration["user_feedback"],
user_feedback_qa=self.current_state.current_iteration["user_feedback_qa"],
docs=self.current_state.docs,
magic_words=magic_words,
next_solution_to_try=None,
test_instructions=json.loads(self.current_state.current_task.get("test_instructions") or "[]"),
)
hunting_cycles = self.current_state.current_iteration.get("bug_hunting_cycles", [])[
0 : (-1 if omit_last_cycle else None)
]
for hunting_cycle in hunting_cycles:
convo = convo.assistant(hunting_cycle["human_readable_instructions"]).template(
"log_data",
backend_logs=hunting_cycle.get("backend_logs"),
frontend_logs=hunting_cycle.get("frontend_logs"),
fix_attempted=hunting_cycle.get("fix_attempted"),
user_feedback=hunting_cycle.get("user_feedback"),
)
if len(convo.messages) > CONVO_ITERATIONS_LIMIT:
convo.slice(1, CONVO_ITERATIONS_LIMIT)
return convo
async def async_task_finish(self):
if self.state_manager.async_tasks:
if not self.state_manager.async_tasks[-1].done():
await self.send_message("Waiting for the bug reproduction instructions...")
await self.state_manager.async_tasks[-1]
self.state_manager.async_tasks = []
def set_data_for_next_hunting_cycle(self, human_readable_instructions, new_status):
self.next_state.current_iteration["description"] = human_readable_instructions
self.next_state.current_iteration["bug_hunting_cycles"] += [
{
"human_readable_instructions": human_readable_instructions,
"fix_attempted": any(
c["fix_attempted"] for c in self.current_state.current_iteration["bug_hunting_cycles"]
),
}
]
self.next_state.current_iteration["status"] = new_status

@ -1,54 +1,17 @@
import asyncio
import re
from enum import Enum
from typing import Optional
from os.path import basename
from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.mixins import FileDiffMixin
from core.agents.response import AgentResponse, ResponseType
from core.config import CODE_MONKEY_AGENT_NAME, DESCRIBE_FILES_AGENT_NAME, IMPLEMENT_CHANGES_AGENT_NAME
from core.config.actions import CM_UPDATE_FILES
from core.db.models import File
from core.llm.convo import Convo
from core.config import DESCRIBE_FILES_AGENT_NAME
from core.llm.parser import JSONParser, OptionalCodeBlockParser
from core.log import get_logger
log = get_logger(__name__)
# Constant for indicating missing new line at the end of a file in a unified diff
NO_EOL = "\\ No newline at end of file"
# Regular expression pattern for matching hunk headers
PATCH_HEADER_PATTERN = re.compile(r"^@@ -(\d+),?(\d+)? \+(\d+),?(\d+)? @@")
# Maximum number of attempts to ask for review if it can't be parsed
MAX_REVIEW_RETRIES = 2
# Maximum number of code implementation attempts after which we accept the changes unconditionally
MAX_CODING_ATTEMPTS = 3
class Decision(str, Enum):
APPLY = "apply"
IGNORE = "ignore"
REWORK = "rework"
class Hunk(BaseModel):
number: int = Field(description="Index of the hunk in the diff. Starts from 1.")
reason: str = Field(description="Reason for applying or ignoring this hunk, or for asking for it to be reworked.")
decision: Decision = Field(description="Whether to apply this hunk, rework, or ignore it.")
class ReviewChanges(BaseModel):
hunks: list[Hunk]
review_notes: str = Field(description="Additional review notes (optional, can be empty).")
class FileDescription(BaseModel):
summary: str = Field(
description="Detailed description summarized what the file is about, and what the major classes, functions, elements or other functionality is implemented."
@ -58,14 +21,7 @@ class FileDescription(BaseModel):
)
def extract_code_blocks(content):
# Use regex to find all <pythagoracode> blocks with file attribute and their content
code_blocks = re.findall(r'<pythagoracode\s+file="(.*?)">(.*?)</pythagoracode>', content, re.DOTALL)
# Convert matches into a list of dictionaries
return [{"file_name": file_name, "file_content": file_content} for file_name, file_content in code_blocks]
class CodeMonkey(FileDiffMixin, BaseAgent):
class CodeMonkey(BaseAgent):
agent_type = "code-monkey"
display_name = "Code Monkey"
@ -73,96 +29,65 @@ class CodeMonkey(FileDiffMixin, BaseAgent):
if self.prev_response and self.prev_response.type == ResponseType.DESCRIBE_FILES:
return await self.describe_files()
else:
data = await self.implement_changes()
if not data:
return AgentResponse.done(self)
return await self.accept_changes(data["path"], data["old_content"], data["new_content"])
return await self.implement_changes()
async def implement_changes(self, data: Optional[dict] = None) -> dict:
async def implement_changes(self) -> AgentResponse:
file_name = self.step["save_file"]["path"]
current_file = await self.state_manager.get_file_by_path(file_name)
file_content = current_file.content.content if current_file else ""
if data is not None:
attempt = data["attempt"] + 1
feedback = data["feedback"]
task = self.current_state.current_task
if self.prev_response and self.prev_response.type == ResponseType.CODE_REVIEW_FEEDBACK:
attempt = self.prev_response.data["attempt"] + 1
feedback = self.prev_response.data["feedback"]
log.debug(f"Fixing file {file_name} after review feedback: {feedback} ({attempt}. attempt)")
await self.ui.send_file_status(file_name, "reworking", source=self.ui_source)
await self.send_message(f"Reworking changes I made to {file_name} ...")
else:
log.debug(f"Implementing file {file_name}")
if data is None:
await self.ui.send_file_status(
file_name, "updating" if file_content else "creating", source=self.ui_source
)
else:
await self.ui.send_file_status(file_name, "reworking", source=self.ui_source)
self.next_state.action = CM_UPDATE_FILES
await self.send_message(f"{'Updating existing' if file_content else 'Creating new'} file {file_name} ...")
self.next_state.action = (
f'Update file "{basename(file_name)}"' if file_content else f'Create file "{basename(file_name)}"'
)
attempt = 1
feedback = None
iterations = self.current_state.iterations
user_feedback = None
user_feedback_qa = None
llm = self.get_llm()
if iterations:
last_iteration = iterations[-1]
instructions = last_iteration.get("description")
user_feedback = last_iteration.get("user_feedback")
user_feedback_qa = last_iteration.get("user_feedback_qa")
instructions = iterations[-1]["description"]
user_feedback = iterations[-1]["user_feedback"]
user_feedback_qa = iterations[-1]["user_feedback_qa"]
else:
instructions = self.current_state.current_task["instructions"]
blocks = extract_code_blocks(instructions)
response = None
if blocks and self.state_manager.get_access_token():
try:
# Try Relace first
block = next((item for item in blocks if item["file_name"] == file_name), None)
if block:
llm = self.get_llm(IMPLEMENT_CHANGES_AGENT_NAME)
convo = Convo().user(
{
"initialCode": file_content,
"editSnippet": block["file_content"],
}
)
response = await llm(convo, temperature=0, parser=OptionalCodeBlockParser())
except Exception:
response = None
# Fall back to OpenAI if Relace wasn't used or returned empty response
if not response or response is None:
llm = self.get_llm(CODE_MONKEY_AGENT_NAME)
convo = AgentConvo(self).template(
"implement_changes",
file_name=file_name,
file_content=file_content,
instructions=instructions,
user_feedback=user_feedback,
user_feedback_qa=user_feedback_qa,
convo = AgentConvo(self).template(
"implement_changes",
file_name=file_name,
file_content=file_content,
instructions=instructions,
user_feedback=user_feedback,
user_feedback_qa=user_feedback_qa,
)
if feedback:
convo.assistant(f"```\n{self.prev_response.data['new_content']}\n```\n").template(
"review_feedback",
content=self.prev_response.data["approved_content"],
original_content=file_content,
rework_feedback=feedback,
)
if feedback:
convo.assistant(f"```\n{data['new_content']}\n```\n").template(
"review_feedback",
content=data["approved_content"],
original_content=file_content,
rework_feedback=feedback,
)
response = await llm(convo, temperature=0, parser=OptionalCodeBlockParser())
return {
"path": file_name,
"old_content": file_content,
"new_content": response,
}
response: str = await llm(convo, temperature=0, parser=OptionalCodeBlockParser())
# FIXME: provide a counter here so that we don't have an endless loop here
return AgentResponse.code_review(self, file_name, task["instructions"], file_content, response, attempt)
async def describe_files(self) -> AgentResponse:
tasks = []
llm = self.get_llm(DESCRIBE_FILES_AGENT_NAME)
to_describe = {
file.path: file.content.content
for file in self.current_state.files
if not file.content.meta.get("description")
file.path: file.content.content for file in self.current_state.files if not file.meta.get("description")
}
for file in self.next_state.files:
@ -171,62 +96,30 @@ class CodeMonkey(FileDiffMixin, BaseAgent):
continue
if content == "":
file.content.meta = {
**file.content.meta,
file.meta = {
**file.meta,
"description": "Empty file",
"references": [],
}
continue
tasks.append(self.describe_file(file, content))
await asyncio.gather(*tasks)
return AgentResponse.done(self)
log.debug(f"Describing file {file.path}")
await self.send_message(f"Describing file {file.path} ...")
async def describe_file(self, file: File, content: str):
"""
Describes a file by sending it to the LLM agent and then updating the file's metadata in the database.
"""
llm = self.get_llm(DESCRIBE_FILES_AGENT_NAME)
log.debug(f"Describing file {file.path}")
convo = (
AgentConvo(self)
.template(
"describe_file",
path=file.path,
content=content,
convo = (
AgentConvo(self)
.template(
"describe_file",
path=file.path,
content=content,
)
.require_schema(FileDescription)
)
.require_schema(FileDescription)
)
llm_response: FileDescription = await llm(convo, parser=JSONParser(spec=FileDescription))
file.content.meta = {
**file.content.meta,
"description": llm_response.summary,
"references": llm_response.references,
}
# ------------------------------
# CODE REVIEW
# ------------------------------
async def accept_changes(self, file_path: str, old_content: str, new_content: str) -> AgentResponse:
await self.ui.send_file_status(file_path, "done", source=self.ui_source)
n_new_lines, n_del_lines = self.get_line_changes(old_content, new_content)
await self.ui.generate_diff(
file_path, old_content, new_content, n_new_lines, n_del_lines, source=self.ui_source
)
llm_response: FileDescription = await llm(convo, parser=JSONParser(spec=FileDescription))
await self.state_manager.save_file(file_path, new_content)
self.step["save_file"]["content"] = new_content
self.next_state.complete_step("save_file")
input_required = self.state_manager.get_input_required(new_content, file_path)
if input_required:
return AgentResponse.input_required(
self,
[{"file": file_path, "line": line} for line in input_required],
)
else:
return AgentResponse.done(self)
file.meta = {
**file.meta,
"description": llm_response.summary,
"references": llm_response.references,
}
return AgentResponse.done(self)

@ -0,0 +1,326 @@
import re
from difflib import unified_diff
from enum import Enum
from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.llm.parser import JSONParser
from core.log import get_logger
log = get_logger(__name__)
# Constant for indicating missing new line at the end of a file in a unified diff
NO_EOL = "\\ No newline at end of file"
# Regular expression pattern for matching hunk headers
PATCH_HEADER_PATTERN = re.compile(r"^@@ -(\d+),?(\d+)? \+(\d+),?(\d+)? @@")
# Maximum number of attempts to ask for review if it can't be parsed
MAX_REVIEW_RETRIES = 2
# Maximum number of code implementation attempts after which we accept the changes unconditionaly
MAX_CODING_ATTEMPTS = 3
class Decision(str, Enum):
APPLY = "apply"
IGNORE = "ignore"
REWORK = "rework"
class Hunk(BaseModel):
number: int = Field(description="Index of the hunk in the diff. Starts from 1.")
reason: str = Field(description="Reason for applying or ignoring this hunk, or for asking for it to be reworked.")
decision: Decision = Field(description="Whether to apply this hunk, rework, or ignore it.")
class ReviewChanges(BaseModel):
hunks: list[Hunk]
review_notes: str = Field(description="Additional review notes (optional, can be empty).")
class CodeReviewer(BaseAgent):
agent_type = "code-reviewer"
display_name = "Code Reviewer"
async def run(self) -> AgentResponse:
if (
not self.prev_response.data["old_content"]
or self.prev_response.data["new_content"] == self.prev_response.data["old_content"]
or self.prev_response.data["attempt"] >= MAX_CODING_ATTEMPTS
):
# we always auto-accept new files and unchanged files, or if we've tried too many times
return await self.accept_changes(self.prev_response.data["path"], self.prev_response.data["new_content"])
approved_content, feedback = await self.review_change(
self.prev_response.data["path"],
self.prev_response.data["instructions"],
self.prev_response.data["old_content"],
self.prev_response.data["new_content"],
)
if feedback:
return AgentResponse.code_review_feedback(
self,
new_content=self.prev_response.data["new_content"],
approved_content=approved_content,
feedback=feedback,
attempt=self.prev_response.data["attempt"],
)
else:
return await self.accept_changes(self.prev_response.data["path"], approved_content)
async def accept_changes(self, path: str, content: str) -> AgentResponse:
await self.state_manager.save_file(path, content)
self.next_state.complete_step()
input_required = self.state_manager.get_input_required(content)
if input_required:
return AgentResponse.input_required(
self,
[{"file": path, "line": line} for line in input_required],
)
else:
return AgentResponse.done(self)
def _get_task_convo(self) -> AgentConvo:
# FIXME: Current prompts reuse conversation from the developer so we have to resort to this
task = self.current_state.current_task
current_task_index = self.current_state.tasks.index(task)
convo = AgentConvo(self).template(
"breakdown",
task=task,
iteration=None,
current_task_index=current_task_index,
)
# TODO: We currently show last iteration to the code monkey; we might need to show the task
# breakdown and all the iterations instead? To think about when refactoring prompts
if self.current_state.iterations:
convo.assistant(self.current_state.iterations[-1]["description"])
else:
convo.assistant(self.current_state.current_task["instructions"])
return convo
async def review_change(
self, file_name: str, instructions: str, old_content: str, new_content: str
) -> tuple[str, str]:
"""
Review changes that were applied to the file.
This asks the LLM to act as a PR reviewer and for each part (hunk) of the
diff, decide if it should be applied (kept) or ignored (removed from the PR).
:param file_name: name of the file being modified
:param instructions: instructions for the reviewer
:param old_content: old file content
:param new_content: new file content (with proposed changes)
:return: tuple with file content update with approved changes, and review feedback
Diff hunk explanation: https://www.gnu.org/software/diffutils/manual/html_node/Hunks.html
"""
hunks = self.get_diff_hunks(file_name, old_content, new_content)
llm = self.get_llm()
convo = (
self._get_task_convo()
.template(
"review_changes",
instructions=instructions,
file_name=file_name,
old_content=old_content,
hunks=hunks,
)
.require_schema(ReviewChanges)
)
llm_response: ReviewChanges = await llm(convo, temperature=0, parser=JSONParser(ReviewChanges))
for i in range(MAX_REVIEW_RETRIES):
reasons = {}
ids_to_apply = set()
ids_to_ignore = set()
ids_to_rework = set()
for hunk in llm_response.hunks:
reasons[hunk.number - 1] = hunk.reason
if hunk.decision == "apply":
ids_to_apply.add(hunk.number - 1)
elif hunk.decision == "ignore":
ids_to_ignore.add(hunk.number - 1)
elif hunk.decision == "rework":
ids_to_rework.add(hunk.number - 1)
n_hunks = len(hunks)
n_review_hunks = len(reasons)
if n_review_hunks == n_hunks:
break
elif n_review_hunks < n_hunks:
error = "Not all hunks have been reviewed. Please review all hunks and add 'apply', 'ignore' or 'rework' decision for each."
elif n_review_hunks > n_hunks:
error = f"Your review contains more hunks ({n_review_hunks}) than in the original diff ({n_hunks}). Note that one hunk may have multiple changed lines."
# Max two retries; if the reviewer still hasn't reviewed all hunks, we'll just use the entire new content
convo.assistant(llm_response.model_dump_json()).user(error)
llm_response = await llm(convo, parser=JSONParser(ReviewChanges))
else:
return new_content, None
hunks_to_apply = [h for i, h in enumerate(hunks) if i in ids_to_apply]
diff_log = f"--- {file_name}\n+++ {file_name}\n" + "\n".join(hunks_to_apply)
hunks_to_rework = [(i, h) for i, h in enumerate(hunks) if i in ids_to_rework]
review_log = (
"\n\n".join([f"## Change\n```{hunk}```\nReviewer feedback:\n{reasons[i]}" for (i, hunk) in hunks_to_rework])
+ "\n\nReview notes:\n"
+ llm_response.review_notes
)
if len(hunks_to_apply) == len(hunks):
await self.send_message("Applying entire change")
log.info(f"Applying entire change to {file_name}")
return new_content, None
elif len(hunks_to_apply) == 0:
if hunks_to_rework:
await self.send_message(
f"Requesting rework for {len(hunks_to_rework)} changes with reason: {llm_response.review_notes}"
)
log.info(f"Requesting rework for {len(hunks_to_rework)} changes to {file_name} (0 hunks to apply)")
return old_content, review_log
else:
# If everything can be safely ignored, it's probably because the files already implement the changes
# from previous tasks (which can happen often). Insisting on a change here is likely to cause problems.
await self.send_message(f"Rejecting entire change with reason: {llm_response.review_notes}")
log.info(f"Rejecting entire change to {file_name} with reason: {llm_response.review_notes}")
return old_content, None
log.debug(f"Applying code change to {file_name}:\n{diff_log}")
new_content = self.apply_diff(file_name, old_content, hunks_to_apply, new_content)
if hunks_to_rework:
log.info(f"Requesting further rework for {len(hunks_to_rework)} changes to {file_name}")
return new_content, review_log
else:
return new_content, None
@staticmethod
def get_diff_hunks(file_name: str, old_content: str, new_content: str) -> list[str]:
"""
Get the diff between two files.
This uses Python difflib to produce an unified diff, then splits
it into hunks that will be separately reviewed by the reviewer.
:param file_name: name of the file being modified
:param old_content: old file content
:param new_content: new file content
:return: change hunks from the unified diff
"""
from_name = "old_" + file_name
to_name = "to_" + file_name
from_lines = old_content.splitlines(keepends=True)
to_lines = new_content.splitlines(keepends=True)
diff_gen = unified_diff(from_lines, to_lines, fromfile=from_name, tofile=to_name)
diff_txt = "".join(diff_gen)
hunks = re.split(r"\n@@", diff_txt, re.MULTILINE)
result = []
for i, h in enumerate(hunks):
# Skip the prologue (file names)
if i == 0:
continue
txt = h.splitlines()
txt[0] = "@@" + txt[0]
result.append("\n".join(txt))
return result
def apply_diff(self, file_name: str, old_content: str, hunks: list[str], fallback: str):
"""
Apply the diff to the original file content.
This uses the internal `_apply_patch` method to apply the
approved diff hunks to the original file content.
If patch apply fails, the fallback is the full new file content
with all the changes applied (as if the reviewer approved everythng).
:param file_name: name of the file being modified
:param old_content: old file content
:param hunks: change hunks from the unified diff
:param fallback: proposed new file content (with all the changes applied)
"""
diff = (
"\n".join(
[
f"--- {file_name}",
f"+++ {file_name}",
]
+ hunks
)
+ "\n"
)
try:
fixed_content = self._apply_patch(old_content, diff)
except Exception as e:
# This should never happen but if it does, just use the new version from
# the LLM and hope for the best
print(f"Error applying diff: {e}; hoping all changes are valid")
return fallback
return fixed_content
# Adapted from https://gist.github.com/noporpoise/16e731849eb1231e86d78f9dfeca3abc (Public Domain)
@staticmethod
def _apply_patch(original: str, patch: str, revert: bool = False):
"""
Apply a patch to a string to recover a newer version of the string.
:param original: The original string.
:param patch: The patch to apply.
:param revert: If True, treat the original string as the newer version and recover the older string.
:return: The updated string after applying the patch.
"""
original_lines = original.splitlines(True)
patch_lines = patch.splitlines(True)
updated_text = ""
index_original = start_line = 0
# Choose which group of the regex to use based on the revert flag
match_index, line_sign = (1, "+") if not revert else (3, "-")
# Skip header lines of the patch
while index_original < len(patch_lines) and patch_lines[index_original].startswith(("---", "+++")):
index_original += 1
while index_original < len(patch_lines):
match = PATCH_HEADER_PATTERN.match(patch_lines[index_original])
if not match:
raise Exception("Bad patch -- regex mismatch [line " + str(index_original) + "]")
line_number = int(match.group(match_index)) - 1 + (match.group(match_index + 1) == "0")
if start_line > line_number or line_number > len(original_lines):
raise Exception("Bad patch -- bad line number [line " + str(index_original) + "]")
updated_text += "".join(original_lines[start_line:line_number])
start_line = line_number
index_original += 1
while index_original < len(patch_lines) and patch_lines[index_original][0] != "@":
if index_original + 1 < len(patch_lines) and patch_lines[index_original + 1][0] == "\\":
line_content = patch_lines[index_original][:-1]
index_original += 2
else:
line_content = patch_lines[index_original]
index_original += 1
if line_content:
if line_content[0] == line_sign or line_content[0] == " ":
updated_text += line_content[1:]
start_line += line_content[0] != line_sign
updated_text += "".join(original_lines[start_line:])
return updated_text

@ -3,7 +3,6 @@ import sys
from copy import deepcopy
from typing import TYPE_CHECKING, Optional
import jsonref
from pydantic import BaseModel
from core.config import get_config
@ -88,48 +87,7 @@ class AgentConvo(Convo):
child.prompt_log = deepcopy(self.prompt_log)
return child
def trim(self, trim_index: int, trim_count: int) -> "AgentConvo":
"""
Trim the conversation starting from the given index by 1 message.
:param trim_index:
:return:
"""
self.messages = self.messages[:trim_index] + self.messages[trim_index + trim_count :]
return self
def slice(self, slice_index: int, slice_count: int) -> "AgentConvo":
"""
Create a new conversation containing messages from slice_index to the end, excluding slice_count messages.
:param slice_index: Starting index to slice from
:param slice_count: Number of messages to exclude from the end
:return: self for method chaining
"""
end_index = max(slice_index, len(self.messages) - slice_count)
self.messages = self.messages[:slice_index] + self.messages[end_index:]
return self
def require_schema(self, model: BaseModel) -> "AgentConvo":
def remove_defs(d):
if isinstance(d, dict):
return {k: remove_defs(v) for k, v in d.items() if k != "$defs"}
elif isinstance(d, list):
return [remove_defs(v) for v in d]
else:
return d
# We want to make the schema as simple as possible to avoid confusing the LLM,
# so we remove (dereference) all the refs we can and show the "final" schema version.
schema_txt = json.dumps(remove_defs(jsonref.loads(json.dumps(model.model_json_schema()))))
self.user(
f"IMPORTANT: Your response MUST conform to this JSON schema:\n```\n{schema_txt}\n```."
f"YOU MUST NEVER add any additional fields to your response, and NEVER add additional preamble like 'Here is your JSON'."
)
return self
def remove_last_x_messages(self, x: int) -> "AgentConvo":
"""
Remove the last `x` messages from the conversation.
"""
self.messages = self.messages[:-x]
schema_txt = json.dumps(model.model_json_schema())
self.user(f"IMPORTANT: Your response MUST conform to this JSON schema:\n```\n{schema_txt}\n```")
return self

@ -1,30 +1,16 @@
import json
from enum import Enum
from typing import Annotated, Literal, Union
from uuid import UUID, uuid4
from typing import Annotated, Literal, Optional, Union
from uuid import uuid4
from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.mixins import ChatWithBreakdownMixin, RelevantFilesMixin
from core.agents.response import AgentResponse
from core.cli.helpers import get_epic_task_number
from core.config import PARSE_TASK_AGENT_NAME, TASK_BREAKDOWN_AGENT_NAME
from core.config.actions import (
DEV_EXECUTE_TASK,
DEV_TASK_BREAKDOWN,
DEV_TASK_REVIEW_FEEDBACK,
DEV_TASK_START,
DEV_TROUBLESHOOT,
DEV_WAIT_TEST,
)
from core.db.models.project_state import IterationStatus, TaskStatus
from core.db.models.specification import Complexity
from core.agents.response import AgentResponse, ResponseType
from core.db.models.project_state import TaskStatus
from core.llm.parser import JSONParser
from core.log import get_logger
from core.telemetry import telemetry
from core.ui.base import ProjectStage, pythagora_source
log = get_logger(__name__)
@ -33,7 +19,6 @@ class StepType(str, Enum):
COMMAND = "command"
SAVE_FILE = "save_file"
HUMAN_INTERVENTION = "human_intervention"
UTILITY_FUNCTION = "utility_function"
class CommandOptions(BaseModel):
@ -61,18 +46,8 @@ class HumanInterventionStep(BaseModel):
human_intervention_description: str
class UtilityFunction(BaseModel):
type: Literal[StepType.UTILITY_FUNCTION] = StepType.UTILITY_FUNCTION
file: str
function_name: str
description: str
return_value: str
input_value: str
status: Literal["mocked", "implemented"]
Step = Annotated[
Union[SaveFileStep, CommandStep, HumanInterventionStep, UtilityFunction],
Union[SaveFileStep, CommandStep, HumanInterventionStep],
Field(discriminator="type"),
]
@ -81,20 +56,17 @@ class TaskSteps(BaseModel):
steps: list[Step]
def has_correct_num_of_tags(response: str) -> bool:
"""
Checks if the response has the correct number of opening and closing tags.
"""
return response.count("<pythagoracode file") == response.count("</pythagoracode>")
class RelevantFiles(BaseModel):
relevant_files: list[str] = Field(description="List of relevant files for the current task.")
class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
class Developer(BaseAgent):
agent_type = "developer"
display_name = "Developer"
async def run(self) -> AgentResponse:
if self.current_state.current_step and self.current_state.current_step.get("type") == "utility_function":
return await self.update_knowledge_base()
if self.prev_response and self.prev_response.type == ResponseType.TASK_REVIEW_FEEDBACK:
return await self.breakdown_current_iteration(self.prev_response.data["feedback"])
if not self.current_state.unfinished_tasks:
log.warning("No unfinished tasks found, nothing to do (why am I called? is this a bug?)")
@ -105,38 +77,28 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
# By default, we want to ask the user if they want to run the task,
# except in certain cases (such as they've just edited it).
# The check for docs is here to prevent us from asking the user whether we should
# run the task twice - we'll only ask if we haven't yet checked for docs.
if not self.current_state.current_task.get("run_always", False) and self.current_state.docs is None:
if not self.current_state.current_task.get("run_always", False):
if not await self.ask_to_execute_task():
return AgentResponse.done(self)
if self.current_state.docs is None and self.current_state.specification.complexity != Complexity.SIMPLE:
# We check for external docs here, to make sure we only fetch the docs
# if the task is actually being done.
return AgentResponse.external_docs_required(self)
return await self.breakdown_current_task()
async def breakdown_current_iteration(self) -> AgentResponse:
async def breakdown_current_iteration(self, review_feedback: Optional[str] = None) -> AgentResponse:
"""
Breaks down current iteration or task review into steps.
:param review_feedback: If provided, the task review feedback is broken down instead of the current iteration
:return: AgentResponse.done(self) when the breakdown is done
"""
current_task = self.current_state.current_task
if self.current_state.current_iteration["status"] in (
IterationStatus.AWAITING_BUG_FIX,
IterationStatus.AWAITING_LOGGING,
):
iteration = self.current_state.current_iteration
description = iteration["bug_hunting_cycles"][-1]["human_readable_instructions"]
user_feedback = iteration["user_feedback"]
source = "bug_hunt"
n_tasks = len(self.next_state.iterations)
log.debug(f"Breaking down the logging cycle {description}")
if review_feedback is not None:
iteration = None
description = review_feedback
user_feedback = ""
source = "review"
n_tasks = 1
log.debug(f"Breaking down the task review feedback {review_feedback}")
await self.send_message("Breaking down the task review feedback...")
else:
iteration = self.current_state.current_iteration
if iteration is None:
@ -148,158 +110,92 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
source = "troubleshooting"
n_tasks = len(self.next_state.iterations)
log.debug(f"Breaking down the iteration {description}")
await self.send_message("Breaking down the current task iteration ...")
if self.current_state.files and self.current_state.relevant_files is None:
await self.get_relevant_files_parallel(user_feedback, description)
await self.get_relevant_files(user_feedback, description)
await self.ui.send_task_progress(
n_tasks, # iterations and reviews can be created only one at a time, so we are always on last one
n_tasks,
current_task["description"],
self.current_state.current_task["description"],
source,
"in-progress",
self.current_state.get_source_index(source),
self.current_state.tasks,
)
llm = self.get_llm(PARSE_TASK_AGENT_NAME)
llm = self.get_llm()
# FIXME: In case of iteration, parse_task depends on the context (files, tasks, etc) set there.
# Ideally this prompt would be self-contained.
# Ideally this prompt would be self-contained.
convo = (
AgentConvo(self).template("parse_task", implementation_instructions=description).require_schema(TaskSteps)
AgentConvo(self)
.template(
"iteration",
current_task=self.current_state.current_task,
user_feedback=user_feedback,
user_feedback_qa=None,
next_solution_to_try=None,
docs=self.current_state.docs,
)
.assistant(description)
.template("parse_task")
.require_schema(TaskSteps)
)
response: TaskSteps = await llm(convo, parser=JSONParser(TaskSteps), temperature=0)
self.set_next_steps(response, source)
if iteration:
if "status" not in iteration or (
iteration["status"] in (IterationStatus.AWAITING_USER_TEST, IterationStatus.AWAITING_BUG_REPRODUCTION)
):
# This is just a support for old iterations that don't have status
self.next_state.complete_iteration()
self.next_state.action = DEV_TROUBLESHOOT.format(len(self.current_state.iterations))
elif iteration["status"] == IterationStatus.IMPLEMENT_SOLUTION:
# If the user requested a change, then, we'll implement it and go straight back to testing
self.next_state.complete_iteration()
self.next_state.action = DEV_TROUBLESHOOT.format(len(self.current_state.iterations))
elif iteration["status"] == IterationStatus.AWAITING_BUG_FIX:
# If bug fixing is done, ask user to test again
self.next_state.action = DEV_WAIT_TEST
self.next_state.current_iteration["status"] = IterationStatus.AWAITING_USER_TEST
elif iteration["status"] == IterationStatus.AWAITING_LOGGING:
# If logging is done, ask user to reproduce the bug
self.next_state.current_iteration["status"] = IterationStatus.AWAITING_BUG_REPRODUCTION
self.next_state.complete_iteration()
self.next_state.action = f"Troubleshooting #{len(self.current_state.iterations)}"
else:
self.next_state.action = DEV_TASK_REVIEW_FEEDBACK
self.next_state.action = "Task review feedback"
current_task_index = self.current_state.tasks.index(current_task)
self.next_state.tasks[current_task_index] = {
**current_task,
}
self.next_state.flag_tasks_as_modified()
return AgentResponse.done(self)
async def breakdown_current_task(self) -> AgentResponse:
current_task = self.current_state.current_task
current_task_index = self.current_state.tasks.index(current_task)
self.next_state.action = DEV_TASK_BREAKDOWN.format(current_task_index + 1)
task = self.current_state.current_task
source = self.current_state.current_epic.get("source", "app")
await self.ui.send_task_progress(
self.current_state.tasks.index(current_task) + 1,
self.current_state.tasks.index(self.current_state.current_task) + 1,
len(self.current_state.tasks),
current_task["description"],
self.current_state.current_task["description"],
source,
"in-progress",
self.current_state.get_source_index(source),
self.current_state.tasks,
)
log.debug(f"Breaking down the current task: {current_task['description']}")
log.debug(f"Breaking down the current task: {task['description']}")
await self.send_message("Thinking about how to implement this task ...")
log.debug(f"Current state files: {len(self.current_state.files)}, relevant {self.current_state.relevant_files}")
# Check which files are relevant to the current task
await self.get_relevant_files_parallel()
current_task_index = self.current_state.tasks.index(current_task)
await self.send_message("### Thinking about how to implement this task ...")
await self.ui.start_breakdown_stream()
await self.ui.set_important_stream()
related_api_endpoints = current_task.get("related_api_endpoints", [])
llm = self.get_llm(TASK_BREAKDOWN_AGENT_NAME, stream_output=True)
# TODO: Temp fix for old projects
if not (
related_api_endpoints
and len(related_api_endpoints) > 0
and all(isinstance(api, dict) and "endpoint" in api for api in related_api_endpoints)
):
related_api_endpoints = []
redo_task_user_feedback = None
if self.current_state.files and self.current_state.relevant_files is None:
await self.get_relevant_files()
if (
self.next_state
and self.next_state.current_task
and self.next_state.current_task.get("redo_human_instructions", None) is not None
):
redo_task_user_feedback = self.next_state.current_task["redo_human_instructions"]
current_task_index = self.current_state.tasks.index(task)
llm = self.get_llm()
convo = AgentConvo(self).template(
"breakdown",
task=current_task,
iteration=None,
current_task_index=current_task_index,
docs=self.current_state.docs,
related_api_endpoints=related_api_endpoints,
redo_task_user_feedback=redo_task_user_feedback,
"breakdown", task=task, iteration=None, current_task_index=current_task_index, docs=self.current_state.docs
)
response: str = await llm(convo)
convo.assistant(response)
max_retries = 2
retry_count = 0
while retry_count < max_retries:
if has_correct_num_of_tags(response):
break
convo.user(
"Ok, now think carefully about your previous response. If the response ends by mentioning something about continuing with the implementation, continue but don't implement any files that have already been implemented. If your last response finishes with an incomplete file, implement that file and any other that needs implementation. Finally, if your last response doesn't end by mentioning continuing and if there isn't an unfinished file implementation, respond only with `DONE` and with nothing else."
)
continue_response: str = await llm(convo)
last_open_tag_index = response.rfind("<pythagoracode file")
response = response[:last_open_tag_index] + continue_response
convo.assistant(response)
retry_count += 1
response = await self.chat_with_breakdown(convo, response)
# FIXME: check if this is correct, as sqlalchemy can't figure out modifications
# to attributes; however, self.next is not saved yet so maybe this is fine
self.next_state.tasks[current_task_index] = {
**current_task,
**task,
"instructions": response,
}
self.next_state.flag_tasks_as_modified()
llm = self.get_llm(PARSE_TASK_AGENT_NAME)
convo = AgentConvo(self).template("parse_task", implementation_instructions=response).require_schema(TaskSteps)
await self.send_message("Breaking down the task into steps ...")
convo.assistant(response).template("parse_task").require_schema(TaskSteps)
response: TaskSteps = await llm(convo, parser=JSONParser(TaskSteps), temperature=0)
# There might be state leftovers from previous tasks that we need to clean here
self.next_state.modified_files = {}
self.set_next_steps(response, source)
self.next_state.current_task["status"] = TaskStatus.IN_PROGRESS
self.next_state.action = DEV_TASK_START.format(current_task_index + 1)
self.next_state.action = f"Task #{current_task_index + 1} start"
await telemetry.trace_code_event(
"task-start",
{
@ -310,10 +206,34 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
)
return AgentResponse.done(self)
async def get_relevant_files(
self, user_feedback: Optional[str] = None, solution_description: Optional[str] = None
) -> AgentResponse:
log.debug("Getting relevant files for the current task")
await self.send_message("Figuring out which project files are relevant for the next task ...")
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
"filter_files",
current_task=self.current_state.current_task,
user_feedback=user_feedback,
solution_description=solution_description,
)
.require_schema(RelevantFiles)
)
llm_response: list[str] = await llm(convo, parser=JSONParser(RelevantFiles), temperature=0)
existing_files = {file.path for file in self.current_state.files}
self.next_state.relevant_files = [path for path in llm_response.relevant_files if path in existing_files]
return AgentResponse.done(self)
def set_next_steps(self, response: TaskSteps, source: str):
# For logging/debugging purposes, we don't want to remove the finished steps
# until we're done with the task.
unique_steps = self.remove_duplicate_steps({**response.model_dump()})
finished_steps = [step for step in self.current_state.steps if step["completed"]]
self.next_state.steps = finished_steps + [
{
@ -321,31 +241,23 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
"completed": False,
"source": source,
"iteration_index": len(self.current_state.iterations),
**step,
**step.model_dump(),
}
for step in unique_steps["steps"]
for step in response.steps
]
if len(self.next_state.unfinished_steps) > 0 and source != "review":
self.next_state.steps += [
# TODO: add refactor step here once we have the refactor agent
{
"id": uuid4().hex,
"completed": False,
"type": "review_task",
"source": source,
"iteration_index": len(self.current_state.iterations),
},
]
log.debug(f"Next steps: {self.next_state.unfinished_steps}")
def remove_duplicate_steps(self, data):
unique_steps = []
# Process steps attribute
for step in data["steps"]:
if isinstance(step, SaveFileStep) and any(
s["type"] == "save_file" and s["save_file"]["path"] == step["save_file"]["path"] for s in unique_steps
):
continue
unique_steps.append(step)
# Update steps attribute
data["steps"] = unique_steps
# Use the serializable_steps for JSON dumping
data["original_response"] = json.dumps(unique_steps, indent=2)
return data
async def ask_to_execute_task(self) -> bool:
"""
Asks the user to approve, skip or edit the current task.
@ -360,83 +272,10 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
buttons["skip"] = "Skip Task"
description = self.current_state.current_task["description"]
epic_index, task_index = get_epic_task_number(self.current_state, self.current_state.current_task)
await self.ui.send_project_stage(
{
"stage": ProjectStage.STARTING_TASK,
"task_index": task_index,
}
)
# find latest finished task, send back logs for it being finished
tasks_done = [task for task in self.current_state.tasks if task not in self.current_state.unfinished_tasks]
previous_task = tasks_done[-1] if tasks_done else None
if previous_task:
e_i, t_i = get_epic_task_number(self.current_state, previous_task)
task_convo = await self.state_manager.get_task_conversation_project_states(
UUID(previous_task["id"]), first_last_only=True
)
await self.ui.send_back_logs(
[
{
"title": previous_task["description"],
"project_state_id": str(task_convo[0].id) if task_convo else "be_0",
"start_id": str(task_convo[0].id) if task_convo else "be_0",
"end_id": str(task_convo[-1].prev_state_id) if task_convo else "be_0",
"labels": [f"E{e_i} / T{t_i}", "Backend", "done"],
}
]
)
await self.ui.send_front_logs_headers(
str(task_convo[0].id) if task_convo else "be_0",
[f"E{e_i} / T{t_i}", "Backend", "done"],
previous_task["description"],
self.current_state.current_task.get("id"),
)
await self.ui.send_front_logs_headers(
str(self.current_state.id),
[f"E{epic_index} / T{task_index}", "Backend", "working"],
description,
self.current_state.current_task.get("id"),
)
await self.ui.send_back_logs(
[
{
"title": description,
"project_state_id": str(self.current_state.id),
"labels": [f"E{epic_index} / T{task_index}", "working"],
}
]
)
await self.ui.clear_main_logs()
await self.send_message(f"Starting task #{task_index} with the description:\n\n## {description}")
if self.current_state.run_command:
await self.ui.send_run_command(self.current_state.run_command)
if self.next_state.current_task.get("redo_human_instructions", None) is not None:
await self.send_message(f"Additional feedback: {self.next_state.current_task['redo_human_instructions']}")
return True
if self.current_state.current_task.get("quick_implementation", False):
return True
if self.current_state.current_task.get("user_added_subsequently", False):
return True
if self.current_state.current_task.get("hardcoded", False):
return True
if self.current_state.current_task and self.current_state.current_task.get("hardcoded", False):
await self.ui.send_message(
"Ok, great, you're now starting to build the backend and the first task is to test how the authentication works. You can now register and login. Your data will be saved into the database.",
source=pythagora_source,
)
await self.send_message("Starting new task with description:")
await self.send_message(description)
user_response = await self.ask_question(
DEV_EXECUTE_TASK,
"Do you want to execute the above task?",
buttons=buttons,
default="yes",
buttons_only=True,
@ -465,8 +304,8 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
initial_text=description,
)
if user_response.button == "cancel" or user_response.cancelled:
# User hasn't edited the task, so we can execute it immediately as is
return await self.ask_to_execute_task()
# User hasn't edited the task so we can execute it immediately as is
return True
self.next_state.current_task["description"] = user_response.text
self.next_state.current_task["run_always"] = True
@ -474,11 +313,3 @@ class Developer(ChatWithBreakdownMixin, RelevantFilesMixin, BaseAgent):
log.info(f"Task description updated to: {user_response.text}")
# Orchestrator will rerun us with the new task description
return False
async def update_knowledge_base(self):
"""
Update the knowledge base with the current task and steps.
"""
await self.state_manager.update_utility_functions(self.current_state.current_step)
self.next_state.complete_step("utility_function")
return AgentResponse.done(self)

@ -3,7 +3,6 @@ from uuid import uuid4
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.db.models.project_state import IterationStatus
from core.log import get_logger
log = get_logger(__name__)
@ -85,7 +84,7 @@ class ErrorHandler(BaseAgent):
log.info("Skipping command error debug (requested by user)")
return AgentResponse.done(self)
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
convo = AgentConvo(self).template(
"debug",
task_steps=self.current_state.steps,
@ -107,12 +106,10 @@ class ErrorHandler(BaseAgent):
{
"id": uuid4().hex,
"user_feedback": f"Error running command: {cmd}",
"user_feedback_qa": None,
"description": llm_response,
"alternative_solutions": [],
"attempts": 1,
"status": IterationStatus.IMPLEMENT_SOLUTION,
"bug_hunting_cycles": [],
"completed": False,
}
]
# TODO: maybe have ProjectState.finished_steps as well? would make the debug/ran_command prompts nicer too

@ -6,7 +6,6 @@ from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.config.actions import EX_RUN_COMMAND, EX_SKIP_COMMAND, RUN_COMMAND
from core.llm.parser import JSONParser
from core.log import get_logger
from core.proc.exec_log import ExecLog
@ -55,6 +54,7 @@ class Executor(BaseAgent):
output_handler=self.output_handler,
exit_handler=self.exit_handler,
)
self.stream_output = True
def for_step(self, step):
# FIXME: not needed, refactor to use self.current_state.current_step
@ -79,39 +79,33 @@ class Executor(BaseAgent):
timeout = options.get("timeout")
if timeout:
q = f"{RUN_COMMAND} {cmd} with {timeout}s timeout?"
q = f"Can I run command: {cmd} with {timeout}s timeout?"
else:
q = f"{RUN_COMMAND} {cmd}?"
q = f"Can I run command: {cmd}?"
confirm = await self.ask_question(
q,
buttons={"yes": "Yes", "no": "No"},
default="yes",
buttons_only=False,
initial_text=cmd,
extra_info={"remove_button": "yes"},
buttons_only=True,
)
if confirm.button == "no":
log.info(f"Skipping command execution of `{cmd}` (requested by user)")
await self.send_message(f"Skipping command {cmd}")
self.complete()
self.next_state.action = EX_SKIP_COMMAND.format(cmd_name)
self.next_state.action = f'Skip "{cmd_name}"'
return AgentResponse.done(self)
if confirm.button != "yes":
cmd = confirm.text
started_at = datetime.now(timezone.utc)
log.info(f"Running command `{cmd}` with timeout {timeout}s")
status_code, stdout, stderr = await self.process_manager.run_command(cmd, timeout=timeout)
llm_response = await self.check_command_output(cmd, timeout, stdout, stderr, status_code)
duration = (datetime.now(timezone.utc) - started_at).total_seconds()
self.complete()
self.next_state.action = EX_RUN_COMMAND.format(cmd_name)
self.next_state.action = f'Run "{cmd_name}"'
exec_log = ExecLog(
started_at=started_at,
@ -128,8 +122,7 @@ class Executor(BaseAgent):
)
await self.state_manager.log_command_run(exec_log)
# FIXME: ErrorHandler isn't debugged with BugHunter - we should move all commands to run before testing and debug them with BugHunter
if True or llm_response.success:
if llm_response.success:
return AgentResponse.done(self)
return AgentResponse.error(
@ -178,4 +171,4 @@ class Executor(BaseAgent):
information we give it.
"""
self.step = None
self.next_state.complete_step("command")
self.next_state.complete_step()

@ -44,9 +44,6 @@ class ExternalDocumentation(BaseAgent):
display_name = "Documentation"
async def run(self) -> AgentResponse:
await self._store_docs([], [])
return AgentResponse.done(self)
if self.current_state.specification.example_project:
log.debug("Example project detected, no documentation selected.")
available_docsets = []
@ -88,7 +85,7 @@ class ExternalDocumentation(BaseAgent):
if not available_docsets:
return {}
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
@ -112,7 +109,7 @@ class ExternalDocumentation(BaseAgent):
queries = {}
await self.send_message("Getting relevant documentation for the following topics:")
for k, short_desc in docsets.items():
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(

@ -1,602 +0,0 @@
import asyncio
import json
import os
import sys
from urllib.parse import urljoin
import httpx
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.git import GitMixin
from core.agents.mixins import FileDiffMixin
from core.agents.response import AgentResponse
from core.cli.helpers import capture_exception
from core.config import FRONTEND_AGENT_NAME, IMPLEMENT_CHANGES_AGENT_NAME, PYTHAGORA_API
from core.config.actions import (
FE_CHANGE_REQ,
FE_CONTINUE,
FE_DONE_WITH_UI,
FE_ITERATION,
FE_ITERATION_DONE,
FE_START,
)
from core.llm.convo import Convo
from core.llm.parser import DescriptiveCodeBlockParser, OptionalCodeBlockParser
from core.log import get_logger
from core.telemetry import telemetry
from core.ui.base import ProjectStage
log = get_logger(__name__)
def has_correct_num_of_backticks(response: str) -> bool:
"""
Checks if the response has the correct number of backticks.
"""
return response.count("```") % 2 == 0 and response.count("```") > 0
class Frontend(FileDiffMixin, GitMixin, BaseAgent):
agent_type = "frontend"
display_name = "Frontend"
async def run(self) -> AgentResponse:
if not self.current_state.epics[-1]["messages"]:
finished = await self.start_frontend()
elif self.next_state.epics[-1].get("file_paths_to_remove_mock"):
finished = await self.remove_mock()
if finished is None:
return AgentResponse.exit(self)
elif not self.next_state.epics[-1].get("fe_iteration_done"):
finished = await self.continue_frontend()
else:
await self.set_app_details()
finished = await self.iterate_frontend()
if finished is None:
return AgentResponse.exit(self)
return await self.end_frontend_iteration(finished)
async def start_frontend(self):
"""
Starts the frontend of the app.
"""
self.state_manager.fe_auto_debug = True
await self.ui.clear_main_logs()
await self.ui.send_front_logs_headers(str(self.next_state.id), ["E2 / T1", "working"], "Building frontend")
await self.ui.send_back_logs(
[
{
"title": "Building frontend",
"project_state_id": str(self.next_state.id),
"labels": ["E2 / T1", "Frontend", "working"],
}
]
)
self.next_state.action = FE_START
await self.send_message("## Building the frontend\n\nThis may take a couple of minutes.")
await self.ui.send_project_stage({"stage": ProjectStage.FRONTEND_STARTED})
await self.ui.set_important_stream(False)
llm = self.get_llm(FRONTEND_AGENT_NAME, stream_output=True)
convo = AgentConvo(self).template(
"build_frontend",
summary=self.state_manager.template["template"].get_summary()
if self.state_manager.template is not None
else self.current_state.specification.template_summary,
description=self.next_state.epics[-1]["description"],
user_feedback=None,
first_time_build=True,
)
response = await llm(convo, parser=DescriptiveCodeBlockParser())
response_blocks = response.blocks
convo.assistant(response.original_response)
# Await the template task if it's not done yet
if self.state_manager.async_tasks:
if not self.state_manager.async_tasks[-1].done():
await self.state_manager.async_tasks[-1]
self.state_manager.async_tasks = []
await self.process_response(response_blocks)
self.next_state.epics[-1]["messages"] = convo.messages
self.next_state.epics[-1]["fe_iteration_done"] = (
"done" in response.original_response[-20:].lower().strip() or len(convo.messages) > 11
)
self.next_state.flag_epics_as_modified()
return False
async def continue_frontend(self):
"""
Continues building the frontend of the app after the initial user input.
"""
self.state_manager.fe_auto_debug = True
self.next_state.action = FE_CONTINUE
await self.ui.send_project_stage({"stage": ProjectStage.CONTINUE_FRONTEND})
await self.send_message("### Continuing to build UI... This may take a couple of minutes")
llm = self.get_llm(FRONTEND_AGENT_NAME, stream_output=True)
convo = AgentConvo(self)
convo.messages = self.current_state.epics[-1]["messages"]
convo.user(
"Ok, now think carefully about your previous response. If the response ends by mentioning something about continuing with the implementation, continue but don't implement any files that have already been implemented. If your last response finishes with an incomplete file, implement that file and any other that needs implementation. Finally, if your last response doesn't end by mentioning continuing and if there isn't an unfinished file implementation, respond only with `DONE` and with nothing else."
)
response = await llm(convo, parser=DescriptiveCodeBlockParser())
response_blocks = response.blocks
convo.assistant(response.original_response)
use_relace = self.current_state.epics[-1].get("use_relace", False)
await self.process_response(response_blocks, relace=use_relace)
if self.next_state.epics[-1].get("manual_iteration", False):
self.next_state.epics[-1]["fe_iteration_done"] = (
has_correct_num_of_backticks(response.original_response)
or self.current_state.epics[-1].get("retry_count", 0) >= 2
)
self.next_state.epics[-1]["retry_count"] = self.current_state.epics[-1].get("retry_count", 0) + 1
else:
self.next_state.epics[-1]["fe_iteration_done"] = (
"done" in response.original_response[-20:].lower().strip() or len(convo.messages) > 15
)
self.next_state.epics[-1]["messages"] = convo.messages
self.next_state.flag_epics_as_modified()
return False
async def iterate_frontend(self):
"""
Iterates over the frontend.
:return: True if the frontend is fully built, False otherwise.
"""
self.next_state.epics[-1]["auto_debug_attempts"] = 0
self.next_state.epics[-1]["retry_count"] = 0
user_input = await self.try_auto_debug()
frontend_only = self.current_state.branch.project.project_type == "swagger"
self.next_state.action = FE_ITERATION
# update the pages in the knowledge base
await self.state_manager.update_implemented_pages_and_apis()
await self.ui.send_project_stage({"stage": ProjectStage.ITERATE_FRONTEND, "iteration_index": 1})
if user_input:
await self.send_message("Errors detected, fixing...")
else:
answer = await self.ask_question(
"Do you want to change anything or report a bug?" if frontend_only else FE_CHANGE_REQ,
buttons={"yes": "I'm done building the UI"} if not frontend_only else None,
default="yes",
extra_info={"restart_app": True, "collect_logs": True},
placeholder='For example, "I don\'t see anything when I open http://localhost:5173/" or "Nothing happens when I click on the NEW PROJECT button"',
)
if answer.button == "yes":
answer = await self.ask_question(
FE_DONE_WITH_UI,
buttons={
"yes": "Yes, let's build the backend",
"no": "No, continue working on the UI",
},
buttons_only=True,
default="yes",
)
if answer.button == "yes":
fe_states = await self.state_manager.get_fe_states()
first_fe_state_id = fe_states[0].id if fe_states else None
last_fe_state_id = fe_states[-1].id if fe_states else None
await self.ui.clear_main_logs()
await self.ui.send_front_logs_headers(
str(first_fe_state_id) if first_fe_state_id else "fe_0",
["E2 / T1", "done"],
"Building frontend",
)
await self.ui.send_back_logs(
[
{
"title": "Building frontend",
"project_state_id": str(first_fe_state_id) if first_fe_state_id else "fe_0",
"start_id": str(first_fe_state_id) if first_fe_state_id else "fe_0",
"end_id": str(last_fe_state_id) if last_fe_state_id else "fe_0",
"labels": ["E2 / T1", "Frontend", "done"],
}
]
)
await self.ui.send_back_logs(
[
{
"title": "Setting up backend",
"disallow_reload": True,
"project_state_id": "be_0",
"labels": ["E2 / T2", "Backend setup", "working"],
}
]
)
await self.ui.send_front_logs_headers("", ["E2 / T2", "working"], "Setting up backend")
return True
elif answer.button == "no":
return False
if answer.text:
user_input = answer.text
await self.send_message("Implementing the changes you suggested...")
llm = self.get_llm(FRONTEND_AGENT_NAME)
relevant_api_documentation = None
if frontend_only:
convo = AgentConvo(self).template(
"is_relevant_for_docs_search",
user_feedback=user_input,
)
response = await llm(convo)
if str(response).lower() == "yes":
error = None
for attempt in range(3):
try:
url = urljoin(PYTHAGORA_API, "rag/search")
async with httpx.AsyncClient(transport=httpx.AsyncHTTPTransport()) as client:
resp = await client.post(
url,
json={"text": user_input, "project_id": str(self.state_manager.project.id)},
headers={"Authorization": f"Bearer {self.state_manager.get_access_token()}"},
)
if resp.status_code in [200]:
relevant_api_documentation = "\n".join(item["content"] for item in resp.json())
break
elif resp.status_code in [401, 403]:
access_token = await self.ui.send_token_expired()
self.state_manager.update_access_token(access_token)
else:
try:
error = resp.json()["error"]
except Exception as e:
error = e
log.warning(f"Failed to fetch from RAG service: {error}")
await self.send_message(
f"Couldn't find any relevant API documentation. Retrying... \nError: {error}"
)
except Exception as e:
error = e
capture_exception(e)
log.warning(f"Failed to fetch from RAG service: {e}", exc_info=True)
if error:
await self.send_message(f"Please try reloading the project. \nError: {error}")
return None
llm = self.get_llm(FRONTEND_AGENT_NAME, stream_output=True)
# try relace first
convo = AgentConvo(self).template(
"iterate_frontend",
description=self.current_state.epics[-1]["description"],
user_feedback=user_input,
relevant_api_documentation=relevant_api_documentation,
first_time_build=False,
)
# replace system prompt because of relace
convo.messages[0]["content"] = AgentConvo(self).render("system_relace")
response = await llm(convo, parser=DescriptiveCodeBlockParser())
relace_finished = await self.process_response(response.blocks, relace=True)
if not relace_finished:
log.debug("Relace didn't finish, reverting to build_frontend")
convo = AgentConvo(self).template(
"build_frontend",
description=self.current_state.epics[-1]["description"],
user_feedback=user_input,
relevant_api_documentation=relevant_api_documentation,
first_time_build=False,
)
response = await llm(convo, parser=DescriptiveCodeBlockParser())
await self.process_response(response.blocks)
convo.assistant(response.original_response)
self.next_state.epics[-1]["messages"] = convo.messages
self.next_state.epics[-1]["use_relace"] = relace_finished
self.next_state.epics[-1]["fe_iteration_done"] = has_correct_num_of_backticks(response.original_response)
self.next_state.epics[-1]["manual_iteration"] = True
self.next_state.flag_epics_as_modified()
return False
async def end_frontend_iteration(self, finished: bool) -> AgentResponse:
"""
Ends the frontend iteration.
:param finished: Whether the frontend is fully built.
:return: AgentResponse.done(self)
"""
if finished:
# TODO Add question if user app is fully finished
self.next_state.action = FE_ITERATION_DONE
self.next_state.complete_epic()
await telemetry.trace_code_event(
"frontend-finished",
{
"description": self.current_state.epics[-1]["description"],
"messages": self.current_state.epics[-1]["messages"],
},
)
if self.state_manager.git_available and self.state_manager.git_used:
await self.git_commit(commit_message="Frontend finished")
inputs = []
for file in self.current_state.files:
if not file.content:
continue
input_required = self.state_manager.get_input_required(file.content.content, file.path)
if input_required:
inputs += [{"file": file.path, "line": line} for line in input_required]
if inputs:
return AgentResponse.input_required(self, inputs)
return AgentResponse.done(self)
async def process_response(self, response_blocks: list, removed_mock: bool = False, relace: bool = False) -> bool:
"""
Processes the response blocks from the LLM.
:param response_blocks: The response blocks from the LLM.
:return: AgentResponse.done(self)
"""
for block in response_blocks:
description = block.description.strip()
content = block.content.strip()
# Split description into lines and check the last line for file path
description_lines = description.split("\n")
last_line = description_lines[-1].strip()
if "file:" in last_line:
# Extract file path from the last line - get everything after "file:"
file_path = last_line[last_line.index("file:") + 5 :].strip()
file_path = file_path.strip("\"'`")
# Skip empty file paths
if file_path.strip() == "":
continue
new_content = content
old_content = self.current_state.get_file_content_by_path(file_path)
if relace:
llm = self.get_llm(IMPLEMENT_CHANGES_AGENT_NAME)
convo = Convo().user(
{
"initialCode": old_content,
"editSnippet": new_content,
}
)
new_content = await llm(convo, temperature=0, parser=OptionalCodeBlockParser())
if not new_content or new_content == ("", 0, 0):
return False
n_new_lines, n_del_lines = self.get_line_changes(old_content, new_content)
await self.ui.send_file_status(file_path, "done", source=self.ui_source)
await self.ui.generate_diff(
file_path, old_content, new_content, n_new_lines, n_del_lines, source=self.ui_source
)
if not removed_mock and self.current_state.branch.project.project_type == "swagger":
if "client/src/api" in file_path:
if not self.next_state.epics[-1].get("file_paths_to_remove_mock"):
self.next_state.epics[-1]["file_paths_to_remove_mock"] = []
self.next_state.epics[-1]["file_paths_to_remove_mock"].append(file_path)
await self.state_manager.save_file(file_path, new_content)
elif "command:" in last_line:
# Split multiple commands and execute them sequentially
commands = content.strip().split("\n")
for command in commands:
command = command.strip()
if command:
# Add "cd client" prefix if not already present
if not command.startswith("cd "):
command = f"cd client && {command}"
if "run start" in command or "run dev" in command:
continue
# if command is cd client && some_command client/ -> won't work, we need to remove client/ after &&
prefix, cmd_part = command.split("&&", 1)
cmd_part = cmd_part.strip().replace("client/", "")
command = f"{prefix} && {cmd_part}"
# check if cmd_part contains npm run something, if that something is not in scripts, then skip it
if "npm run" in cmd_part:
npm_script = cmd_part.split("npm run")[1].strip()
absolute_path = os.path.join(
self.state_manager.get_full_project_root(),
os.path.join(
"client" if "client" in prefix else "server" if "server" in prefix else "",
"package.json",
),
)
with open(absolute_path, "r") as file:
package_json = json.load(file)
if npm_script not in package_json.get("scripts", {}):
log.warning(
f"Skipping command: {command} as npm script {npm_script} not found, command is {command}"
)
continue
await self.send_message(f"Running command: `{command}`...")
await self.process_manager.run_command(command)
else:
log.info(f"Unknown block description: {description}")
return True
async def remove_mock(self):
"""
Remove mock API from the backend and replace it with api endpoints defined in the external documentation
"""
new_file_paths = self.current_state.epics[-1]["file_paths_to_remove_mock"]
llm = self.get_llm(FRONTEND_AGENT_NAME)
for file_path in new_file_paths:
old_content = self.current_state.get_file_content_by_path(file_path)
convo = AgentConvo(self).template("create_rag_query", file_content=old_content)
topics = await llm(convo)
if topics != "None":
error = None
for attempt in range(3):
try:
url = urljoin(PYTHAGORA_API, "rag/search")
async with httpx.AsyncClient(transport=httpx.AsyncHTTPTransport()) as client:
resp = await client.post(
url,
json={"text": topics, "project_id": str(self.state_manager.project.id)},
headers={"Authorization": f"Bearer {self.state_manager.get_access_token()}"},
)
if resp.status_code == 200:
resp_json = resp.json()
relevant_api_documentation = "\n".join(item["content"] for item in resp_json)
referencing_files = await self.state_manager.get_referencing_files(
self.current_state, file_path
)
convo = AgentConvo(self).template(
"remove_mock",
relevant_api_documentation=relevant_api_documentation,
file_content=old_content,
file_path=file_path,
referencing_files=referencing_files,
lines=len(old_content.splitlines()),
)
response = await llm(convo, parser=DescriptiveCodeBlockParser())
response_blocks = response.blocks
convo.assistant(response.original_response)
await self.process_response(response_blocks, removed_mock=True)
self.next_state.epics[-1]["file_paths_to_remove_mock"].remove(file_path)
break
elif resp.status_code in [401, 403]:
access_token = await self.ui.send_token_expired()
self.state_manager.update_access_token(access_token)
else:
try:
error = resp.json()["error"]
except Exception as e:
error = e
log.warning(f"Failed to fetch from RAG service: {error}")
await self.send_message(
f"I couldn't find any relevant API documentation. Retrying... \nError: {error}"
)
except Exception as e:
capture_exception(e)
log.warning(f"Failed to fetch from RAG service: {e}", exc_info=True)
if error:
await self.send_message(f"Please try reloading the project. \nError: {error}")
return None
return False
async def set_app_details(self):
"""
Sets the app details.
"""
command = "npm run start"
app_link = "http://localhost:5173"
self.next_state.run_command = command
# todo store app link and send whenever we are sending run_command
# self.next_state.app_link = app_link
await self.ui.send_run_command(command)
await self.ui.send_app_link(app_link)
async def kill_app(self):
is_win = sys.platform.lower().startswith("win")
# TODO make ports configurable
# kill frontend - both swagger and node
if is_win:
await self.process_manager.run_command(
"""for /f "tokens=5" %a in ('netstat -ano ^| findstr :5173 ^| findstr LISTENING') do taskkill /F /PID %a""",
show_output=False,
)
else:
await self.process_manager.run_command("lsof -ti:5173 | xargs -r kill", show_output=False)
# if node project, kill backend as well
if self.state_manager.project.project_type == "node":
if is_win:
await self.process_manager.run_command(
"""for /f "tokens=5" %a in ('netstat -ano ^| findstr :3000 ^| findstr LISTENING') do taskkill /F /PID %a""",
show_output=False,
)
else:
await self.process_manager.run_command("lsof -ti:3000 | xargs -r kill", show_output=False)
async def try_auto_debug(self) -> str:
if not self.state_manager.fe_auto_debug:
self.state_manager.fe_auto_debug = True
return ""
if self.next_state.epics[-1].get("auto_debug_attempts", 0) >= 3:
return ""
count = 3
try:
await self.send_message(
f"### Auto-debugging the frontend #{self.next_state.epics[-1]['auto_debug_attempts']+1}"
)
self.next_state.epics[-1]["auto_debug_attempts"] = (
self.current_state.epics[-1].get("auto_debug_attempts", 0) + 1
)
# kill app
await self.kill_app()
npm_proc = await self.process_manager.start_process("npm run start &", show_output=False)
while True:
if count == 3:
await asyncio.sleep(5)
else:
await asyncio.sleep(2)
diff_stdout, diff_stderr = await npm_proc.read_output()
if (diff_stdout == "" and diff_stderr == "") or count <= 0:
break
count -= 1
await self.process_manager.run_command("curl http://localhost:5173", show_output=False)
await asyncio.sleep(1)
diff_stdout, diff_stderr = await npm_proc.read_output()
# kill app again
await self.kill_app()
if diff_stdout or diff_stderr:
await self.send_message(f"### Auto-debugging found an error: \n{diff_stdout}\n{diff_stderr}")
log.debug(f"Auto-debugging output:\n{diff_stdout}\n{diff_stderr}")
return f"I got an error. Here are the logs:\n{diff_stdout}\n{diff_stderr}"
except Exception as e:
capture_exception(e)
log.error(f"Error during auto-debugging: {e}", exc_info=True)
await self.send_message("### All good, no errors found.")
return ""

@ -1,167 +0,0 @@
import os
from typing import Optional
from core.agents.convo import AgentConvo
from core.config.magic_words import GITIGNORE_CONTENT
from core.ui.base import pythagora_source
class GitMixin:
"""
Mixin class for git commands
"""
async def check_git_installed(self) -> bool:
"""Check if git is installed on the system."""
status_code, _, _ = await self.process_manager.run_command("git --version", show_output=False)
git_available = status_code == 0
self.state_manager.git_available = git_available
return git_available
async def is_git_initialized(self) -> bool:
"""Check if git is initialized in the workspace."""
workspace_path = self.state_manager.get_full_project_root()
status_code, _, _ = await self.process_manager.run_command(
"git rev-parse --git-dir --is-inside-git-dir",
cwd=workspace_path,
show_output=False,
)
# Will return status code 0 only if .git exists in the current directory
git_used = status_code == 0 and os.path.exists(os.path.join(workspace_path, ".git"))
self.state_manager.git_used = git_used
return git_used
async def init_git_if_needed(self) -> bool:
"""
Initialize git repository if it hasn't been initialized yet.
Returns True if initialization was needed and successful.
"""
workspace_path = self.state_manager.get_full_project_root()
if await self.is_git_initialized():
return False
answer = await self.ui.ask_question(
"Git is not initialized for this project. Do you want to initialize it now?",
buttons={"yes": "Yes", "no": "No"},
default="yes",
buttons_only=True,
source=pythagora_source,
)
if answer.button == "no":
return False
else:
status_code, _, stderr = await self.process_manager.run_command("git init", cwd=workspace_path)
if status_code != 0:
raise RuntimeError(f"Failed to initialize git repository: {stderr}")
gitignore_path = os.path.join(workspace_path, ".gitignore")
try:
with open(gitignore_path, "w") as f:
f.write(GITIGNORE_CONTENT)
except Exception as e:
raise RuntimeError(f"Failed to create .gitignore file: {str(e)}")
# First check if there are any changes to commit
status_code, stdout, stderr = await self.process_manager.run_command(
"git status --porcelain",
cwd=workspace_path,
)
if status_code == 0 and stdout.strip(): # If there are changes (stdout is not empty)
# Stage all files
status_code, _, stderr = await self.process_manager.run_command(
"git add .",
cwd=workspace_path,
)
if status_code != 0:
raise RuntimeError(f"Failed to stage files: {stderr}")
# Create initial commit
status_code, _, stderr = await self.process_manager.run_command(
'git commit -m "initial commit"', cwd=workspace_path
)
if status_code != 0:
raise RuntimeError(f"Failed to create initial commit: {stderr}")
self.state_manager.git_used = True
return True
async def git_commit(self, commit_message: Optional[str] = None) -> None:
"""
Create a git commit with the specified message. Commit message is optional.
Raises RuntimeError if the commit fails.
"""
workspace_path = self.state_manager.get_full_project_root()
# Check if there are any changes to commit
status_code, git_status, stderr = await self.process_manager.run_command(
"git status --porcelain",
cwd=workspace_path,
show_output=False,
)
if status_code != 0:
raise RuntimeError(f"Failed to get git status: {stderr}")
if not git_status.strip():
return
answer = await self.ui.ask_question(
"Do you want to create new git commit?",
buttons={"yes": "Yes", "no": "No"},
default="yes",
buttons_only=True,
source=pythagora_source,
)
if answer.button == "no":
return
# Stage all changes
status_code, _, stderr = await self.process_manager.run_command("git add .", cwd=workspace_path)
if status_code != 0:
raise RuntimeError(f"Failed to stage changes: {stderr}")
# Get git diff
status_code, git_diff, stderr = await self.process_manager.run_command(
"git diff --cached || git diff",
cwd=workspace_path,
show_output=False,
)
if status_code != 0:
raise RuntimeError(f"Failed to create initial commit: {stderr}")
if not commit_message:
llm = self.get_llm()
convo = AgentConvo(self).template(
"commit",
git_diff=git_diff,
)
commit_message: str = await llm(convo)
answer = await self.ui.ask_question(
f"Do you accept this 'git commit' message? Here is suggested message: '{commit_message}'",
buttons={"yes": "Yes", "edit": "Edit", "no": "No, I don't want to commit changes."},
default="yes",
buttons_only=True,
source=pythagora_source,
)
if answer.button == "no":
return
elif answer.button == "edit":
user_message = await self.ui.ask_question(
"Please enter the commit message",
source=pythagora_source,
initial_text=commit_message,
)
commit_message = user_message.text
# Create commit
status_code, _, stderr = await self.process_manager.run_command(
f'git commit -m "{commit_message}"', cwd=workspace_path
)
if status_code != 0:
raise RuntimeError(f"Failed to create commit: {stderr}")

@ -1,6 +1,5 @@
from core.agents.base import BaseAgent
from core.agents.response import AgentResponse, ResponseType
from core.config.actions import CONTINUE_WHEN_DONE, HUMAN_INTERVENTION_QUESTION
class HumanInput(BaseAgent):
@ -16,14 +15,13 @@ class HumanInput(BaseAgent):
async def human_intervention(self, step) -> AgentResponse:
description = step["human_intervention_description"]
await self.send_message(f"## {HUMAN_INTERVENTION_QUESTION}\n\n{description}")
await self.ask_question(
CONTINUE_WHEN_DONE,
f"I need human intervention: {description}",
buttons={"continue": "Continue"},
default="continue",
buttons_only=True,
)
self.next_state.complete_step("human_intervention")
self.next_state.complete_step()
return AgentResponse.done(self)
async def input_required(self, files: list[dict]) -> AgentResponse:
@ -37,5 +35,12 @@ class HumanInput(BaseAgent):
# figure out where its local files are and how to open it.
full_path = self.state_manager.file_system.get_full_path(file)
await self.ui.open_editor(full_path, line, True)
await self.send_message(f"Input required on {file}:{line}")
await self.ui.open_editor(full_path, line)
await self.ask_question(
f"Please open {file} and modify line {line} according to the instructions.",
buttons={"continue": "Continue"},
default="continue",
buttons_only=True,
)
return AgentResponse.done(self)

@ -54,7 +54,7 @@ class Importer(BaseAgent):
await self.state_manager.commit()
async def analyze_project(self):
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
self.send_message("Inspecting most important project files ...")

@ -1,14 +0,0 @@
from core.agents.base import BaseAgent
from core.agents.response import AgentResponse
class LegacyHandler(BaseAgent):
agent_type = "legacy-handler"
display_name = "Legacy Handler"
async def run(self) -> AgentResponse:
if self.data["type"] == "review_task":
self.next_state.complete_step("review_task")
return AgentResponse.done(self)
raise ValueError(f"Unknown reason for calling Legacy Handler with data: {self.data}")

@ -1,82 +1,6 @@
import asyncio
import json
from typing import List, Optional
from pydantic import BaseModel, Field
from typing import Optional
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.cli.helpers import get_line_changes
from core.config import GET_RELEVANT_FILES_AGENT_NAME, TASK_BREAKDOWN_AGENT_NAME, TROUBLESHOOTER_BUG_REPORT
from core.config.actions import MIX_BREAKDOWN_CHAT_PROMPT
from core.config.constants import CONVO_ITERATIONS_LIMIT
from core.config.magic_words import ALWAYS_RELEVANT_FILES
from core.llm.parser import JSONParser
from core.log import get_logger
from core.ui.base import ProjectStage
log = get_logger(__name__)
class RelevantFiles(BaseModel):
relevant_files: Optional[List[str]] = Field(
description="List of files you want to add to the list of relevant files."
)
class Test(BaseModel):
title: str = Field(description="Very short title of the test.")
action: str = Field(description="More detailed description of what actions have to be taken to test the app.")
result: str = Field(description="Expected result that verifies successful test.")
class TestSteps(BaseModel):
steps: List[Test]
class ChatWithBreakdownMixin:
"""
Provides a method to chat with the user and provide a breakdown of the conversation.
"""
async def chat_with_breakdown(self, convo: AgentConvo, breakdown: str) -> AgentConvo:
"""
Chat with the user and provide a breakdown of the conversation.
:param convo: The conversation object.
:param breakdown: The breakdown of the conversation.
:return: The breakdown.
"""
llm = self.get_llm(TASK_BREAKDOWN_AGENT_NAME, stream_output=True)
while True:
await self.ui.send_project_stage(
{
"stage": ProjectStage.BREAKDOWN_CHAT,
"agent": self.agent_type,
}
)
if self.state_manager.auto_confirm_breakdown:
break
chat = await self.ask_question(
MIX_BREAKDOWN_CHAT_PROMPT,
buttons={"yes": "Yes, looks good!"},
default="yes",
verbose=False,
)
if chat.button == "yes":
break
if len(convo.messages) > CONVO_ITERATIONS_LIMIT:
convo.slice(3, CONVO_ITERATIONS_LIMIT)
convo.user(chat.text)
breakdown: str = await llm(convo)
convo.assistant(breakdown)
return breakdown
class IterationPromptMixin:
@ -92,7 +16,6 @@ class IterationPromptMixin:
*,
user_feedback_qa: Optional[list[str]] = None,
next_solution_to_try: Optional[str] = None,
bug_hunting_cycles: Optional[dict] = None,
) -> str:
"""
Generate a new solution for the problem the user reported.
@ -100,109 +23,15 @@ class IterationPromptMixin:
:param user_feedback: User feedback about the problem.
:param user_feedback_qa: Additional q/a about the problem provided by the user (optional).
:param next_solution_to_try: Hint from ProblemSolver on which solution to try (optional).
:param bug_hunting_cycles: Data about logs that need to be added to the code (optional).
:return: The generated solution to the problem.
"""
llm = self.get_llm(TROUBLESHOOTER_BUG_REPORT, stream_output=True)
llm = self.get_llm()
convo = AgentConvo(self).template(
"iteration",
current_task=self.current_state.current_task,
user_feedback=user_feedback,
user_feedback_qa=user_feedback_qa,
next_solution_to_try=next_solution_to_try,
bug_hunting_cycles=bug_hunting_cycles,
test_instructions=json.loads(self.current_state.current_task.get("test_instructions", "[]")),
)
llm_solution: str = await llm(convo)
llm_solution = await self.chat_with_breakdown(convo, llm_solution)
return llm_solution
class RelevantFilesMixin:
"""
Asynchronously retrieves relevant files for the current task by separating front-end and back-end files, and processing them in parallel.
This method initiates two asynchronous tasks to fetch relevant files for the front-end (client) and back-end (server) respectively.
It then combines the results, filters out any non-existing files, and updates the current and next state with the relevant files.
"""
async def get_relevant_files_parallel(
self, user_feedback: Optional[str] = None, solution_description: Optional[str] = None
) -> AgentResponse:
tasks = [
self.get_relevant_files(
user_feedback=user_feedback, solution_description=solution_description, dir_type="client"
),
self.get_relevant_files(
user_feedback=user_feedback, solution_description=solution_description, dir_type="server"
),
]
responses = await asyncio.gather(*tasks)
relevant_files = [item for sublist in responses for item in sublist]
existing_files = {file.path for file in self.current_state.files}
relevant_files = [path for path in relevant_files if path in existing_files]
self.current_state.relevant_files = relevant_files
self.next_state.relevant_files = relevant_files
return AgentResponse.done(self)
async def get_relevant_files(
self,
user_feedback: Optional[str] = None,
solution_description: Optional[str] = None,
dir_type: Optional[str] = None,
) -> list[str]:
log.debug(
"Getting relevant files for the current task for: " + ("frontend" if dir_type == "client" else "backend")
)
relevant_files = set()
llm = self.get_llm(GET_RELEVANT_FILES_AGENT_NAME)
convo = (
AgentConvo(self)
.template(
"filter_files",
user_feedback=user_feedback,
solution_description=solution_description,
relevant_files=relevant_files,
dir_type=dir_type,
)
.require_schema(RelevantFiles)
)
llm_response: RelevantFiles = await llm(convo, parser=JSONParser(RelevantFiles), temperature=0)
existing_files = {file.path for file in self.current_state.files}
if not llm_response.relevant_files:
return []
paths_list = [path for path in llm_response.relevant_files if path in existing_files]
try:
for file_path in ALWAYS_RELEVANT_FILES:
if file_path not in paths_list and file_path in existing_files:
paths_list.append(file_path)
except Exception as e:
log.error(f"Error while getting most important files: {e}")
return paths_list
class FileDiffMixin:
"""
Provides a method to generate a diff between two files.
"""
def get_line_changes(self, old_content: str, new_content: str) -> tuple[int, int]:
"""
Get the number of added and deleted lines between two files.
This uses Python difflib to produce a unified diff, then counts
the number of added and deleted lines.
:param old_content: old file content
:param new_content: new file content
:return: a tuple (n_new_lines, n_del_lines)
"""
return get_line_changes(old_content, new_content)

@ -1,39 +1,33 @@
import asyncio
import json
import os
import re
from typing import List, Optional, Union
from typing import Optional
from core.agents.architect import Architect
from core.agents.base import BaseAgent
from core.agents.bug_hunter import BugHunter
from core.agents.code_monkey import CodeMonkey
from core.agents.code_reviewer import CodeReviewer
from core.agents.developer import Developer
from core.agents.error_handler import ErrorHandler
from core.agents.executor import Executor
from core.agents.external_docs import ExternalDocumentation
from core.agents.frontend import Frontend
from core.agents.git import GitMixin
from core.agents.human_input import HumanInput
from core.agents.importer import Importer
from core.agents.legacy_handler import LegacyHandler
from core.agents.problem_solver import ProblemSolver
from core.agents.response import AgentResponse, ResponseType
from core.agents.spec_writer import SpecWriter
from core.agents.task_completer import TaskCompleter
from core.agents.task_reviewer import TaskReviewer
from core.agents.tech_lead import TechLead
from core.agents.tech_writer import TechnicalWriter
from core.agents.troubleshooter import Troubleshooter
from core.agents.wizard import Wizard
from core.db.models.project_state import IterationStatus, TaskStatus
from core.db.models.project_state import TaskStatus
from core.db.models.specification import Complexity
from core.log import get_logger
from core.telemetry import telemetry
from core.ui.base import UserInterruptError
from core.ui.base import ProjectStage
log = get_logger(__name__)
class Orchestrator(BaseAgent, GitMixin):
class Orchestrator(BaseAgent):
"""
Main agent that controls the flow of the process.
@ -62,89 +56,15 @@ class Orchestrator(BaseAgent, GitMixin):
await self.init_ui()
await self.offline_changes_check()
await self.install_dependencies()
if self.args.use_git and await self.check_git_installed():
await self.init_git_if_needed()
await self.set_frontend_script()
await self.set_package_json()
await self.set_vite_config()
await self.set_favicon()
await self.enable_debugger()
await self.ui.knowledge_base_update(
{
"pages": self.current_state.knowledge_base.pages,
"apis": self.current_state.knowledge_base.apis,
"user_options": self.current_state.knowledge_base.user_options,
"utility_functions": self.current_state.knowledge_base.utility_functions,
}
)
# TODO: consider refactoring this into two loop; the outer with one iteration per committed step,
# TODO: consider refactoring this into two loop; the outer with one iteration per comitted step,
# and the inner which runs the agents for the current step until they're done. This would simplify
# handle_done() and let us do other per-step processing (eg. describing files) in between agent runs.
while True:
# If the task is marked as "redo_human_instructions", we need to reload the project at the state before the current task breakdown
if (
self.current_state.current_task
and self.current_state.current_task.get("redo_human_instructions", None) is not None
):
redo_human_instructions = self.current_state.current_task["redo_human_instructions"]
project_state = await self.state_manager.get_project_state_for_redo_task(self.current_state)
if project_state is not None:
await self.state_manager.load_project(
branch_id=project_state.branch_id, step_index=project_state.step_index
)
await self.state_manager.restore_files()
self.current_state.epics[-1]["completed"] = False
self.next_state.epics[-1]["completed"] = False
self.next_state.current_task["redo_human_instructions"] = redo_human_instructions
await self.update_stats()
agent = self.create_agent(response)
# In case where agent is a list, run all agents in parallel.
# Only one agent type can be run in parallel at a time (for now). See handle_parallel_responses().
if isinstance(agent, list):
tasks = [single_agent.run() for single_agent in agent]
log.debug(
f"Running agents {[a.__class__.__name__ for a in agent]} (step {self.current_state.step_index})"
)
responses = await asyncio.gather(*tasks)
response = self.handle_parallel_responses(agent[0], responses)
should_update_knowledge_base = any(
"src/pages/" in single_agent.step.get("save_file", {}).get("path", "")
or "src/api/" in single_agent.step.get("save_file", {}).get("path", "")
or single_agent.current_state.current_task.get("related_api_endpoints")
for single_agent in agent
)
if should_update_knowledge_base:
files_with_implemented_apis = [
{
"path": single_agent.step.get("save_file", {}).get("path", None),
"content": single_agent.step.get("save_file", {}).get("content", None),
"related_api_endpoints": single_agent.current_state.current_task.get(
"related_api_endpoints"
),
"line": 0,
}
for single_agent in agent
if single_agent.current_state.current_task.get("related_api_endpoints")
]
await self.state_manager.update_apis(files_with_implemented_apis)
await self.state_manager.update_implemented_pages_and_apis()
else:
log.debug(f"Running agent {agent.__class__.__name__} (step {self.current_state.step_index})")
try:
response = await agent.run()
except UserInterruptError:
log.debug("User interrupted the agent!")
response = AgentResponse.done(self)
agent = self.create_agent(response)
log.debug(f"Running agent {agent.__class__.__name__} (step {self.current_state.step_index})")
response = await agent.run()
if response.type == ResponseType.EXIT:
log.debug(f"Agent {agent.__class__.__name__} requested exit")
@ -152,221 +72,11 @@ class Orchestrator(BaseAgent, GitMixin):
if response.type == ResponseType.DONE:
response = await self.handle_done(agent, response)
log.debug(f"Agent {agent.__class__.__name__} returned")
if not isinstance(agent, list) and agent.agent_type == "spec-writer":
project_details = self.state_manager.get_project_info()
await self.ui.send_project_info(
project_details["name"],
project_details["id"],
project_details["folderName"],
project_details["createdAt"],
)
continue
# TODO: rollback changes to "next" so they aren't accidentally committed?
return True
async def install_dependencies(self):
# First check if package.json exists
package_json_path = os.path.join(self.state_manager.get_full_project_root(), "package.json")
if not os.path.exists(package_json_path):
# Skip if no package.json found
return
# Then check if node_modules directory exists
node_modules_path = os.path.join(self.state_manager.get_full_project_root(), "node_modules")
if not os.path.exists(node_modules_path):
await self.send_message("Installing project dependencies...")
await self.process_manager.run_command("npm install", show_output=False, timeout=600)
async def set_frontend_script(self):
file_path = os.path.join("client", "index.html")
absolute_path = os.path.join(self.state_manager.get_full_project_root(), file_path)
script_tag = '<script src="https://s3.us-east-1.amazonaws.com/assets.pythagora.ai/scripts/utils.js"></script>'
# Check if file exists
if not os.path.exists(absolute_path):
return
try:
# Read the HTML file
with open(absolute_path, "r", encoding="utf-8") as file:
content = file.read()
# Check if script already exists
if script_tag in content:
return
# Find the head tag and title tag
head_match = re.search(r"<head[^>]*>(.*?)</head>", content, re.DOTALL | re.IGNORECASE)
if head_match:
head_content = head_match.group(1)
title_match = re.search(r"(<title[^>]*>.*?</title>)", head_content, re.DOTALL | re.IGNORECASE)
if title_match:
# Insert after title
new_head = head_content.replace(title_match.group(1), f"{title_match.group(1)}\n {script_tag}")
else:
# Insert at the beginning of head
new_head = f"\n {script_tag}{head_content}"
# Replace old head content with new one
new_content = content.replace(head_content, new_head)
await self.state_manager.save_file(file_path, new_content)
except Exception as e:
log.error(f"An error occurred: {str(e)}")
async def enable_debugger(self):
absolute_path = os.path.join(self.state_manager.get_full_project_root(), "package.json")
if not os.path.exists(absolute_path):
return
try:
with open(absolute_path, "r") as file:
package_json = json.load(file)
if "debug" not in package_json["scripts"]:
package_json["scripts"]["debug"] = (
'concurrently -n "client,server" "npm run client" "cross-env NODE_OPTIONS=--inspect-brk=9229 npm run server"'
)
if "devDependencies" not in package_json:
package_json["devDependencies"] = {}
if "cross-env" not in package_json["devDependencies"]:
package_json["devDependencies"]["cross-env"] = "^7.0.3"
else:
return
await self.state_manager.save_file(absolute_path, json.dumps(package_json))
await self.process_manager.run_command("npm install", show_output=True, timeout=600)
log.debug("Debugger support added.")
except Exception as e:
log.debug(f"An error occurred: {e}")
async def set_favicon(self):
"""
Set up favicon link in the client/index.html file.
"""
try:
client_dir = os.path.join(self.state_manager.get_full_project_root(), "client")
index_path = os.path.join(client_dir, "index.html")
if not os.path.exists(index_path):
return
# Read the HTML file
with open(index_path, "r", encoding="utf-8") as file:
content = file.read()
favicon_link = '<link rel="icon" type="image/x-icon" href="https://s3.us-east-1.amazonaws.com/assets.pythagora.ai/logos/favicon.ico" />'
# Check if favicon link already exists
if favicon_link in content:
return
# Find the position where to insert the favicon link
# Look for </head> tag and insert before it
updated_content = content.replace("</head>", f" {favicon_link}\n </head>")
await self.state_manager.save_file(index_path, updated_content)
log.debug("Favicon link added to index.html")
except Exception as e:
log.debug(f"An error occurred while setting favicon: {e}")
async def set_package_json(self):
file_path = os.path.join("client", "package.json")
absolute_path = os.path.join(self.state_manager.get_full_project_root(), file_path)
if not os.path.exists(absolute_path):
return
try:
script = "vite build"
with open(absolute_path, "r") as file:
package_json = json.load(file)
if package_json["scripts"].get("build") == script:
return
package_json["scripts"]["build"] = script
await self.state_manager.save_file(absolute_path, json.dumps(package_json, indent=4))
log.debug(f"Build script changed to {script}.")
except Exception as e:
log.debug(f"An error occurred: {e}")
async def set_vite_config(self):
file_path = os.path.join("client", "vite.config.ts")
absolute_path = os.path.join(self.state_manager.get_full_project_root(), file_path)
if not os.path.exists(absolute_path):
return
try:
# Read the current file
with open(absolute_path, "r", encoding="utf-8") as file:
current_content = file.read()
# Check if required configs already exist
has_host_true = "host: true" in current_content
has_watch_config = "watch: {" in current_content and "ignored:" in current_content
# If both required configs exist, no need to change anything
if has_host_true and has_watch_config:
log.debug("Vite config already has host:true and watch configuration. No changes needed.")
return
# Get the template path
project_root = self.state_manager.get_full_project_root()
base_path = project_root.split("/pythagora-core")[0] + "/pythagora-core"
template_path = os.path.join(
base_path, "core", "templates", "tree", "vite_react", "client", "vite.config.ts"
)
# Read the template file
with open(template_path, "r", encoding="utf-8") as file:
template_content = file.read()
# Save the template content to the target file
await self.state_manager.save_file(file_path, template_content)
log.debug("Updated vite.config.ts with the template configuration.")
except Exception as e:
log.debug(f"An error occurred while updating vite.config.ts: {e}")
def handle_parallel_responses(self, agent: BaseAgent, responses: List[AgentResponse]) -> AgentResponse:
"""
Handle responses from agents that were run in parallel.
This method is called when multiple agents are run in parallel, and it
should return a single response that represents the combined responses
of all agents.
:param agent: The original agent that was run in parallel.
:param responses: List of responses from all agents.
:return: Combined response.
"""
response = AgentResponse.done(agent)
if isinstance(agent, CodeMonkey):
files = []
for single_response in responses:
if single_response.type == ResponseType.INPUT_REQUIRED:
files += single_response.data.get("files", [])
break
if files:
response = AgentResponse.input_required(agent, files)
return response
else:
raise ValueError(f"Unhandled parallel agent type: {agent.__class__.__name__}")
async def offline_changes_check(self):
"""
Check for changes outside Pythagora.
@ -375,45 +85,40 @@ class Orchestrator(BaseAgent, GitMixin):
import if needed.
"""
try:
log.info("Checking for offline changes.")
modified_files = await self.state_manager.get_modified_files_with_content()
log.info("Checking for offline changes.")
modified_files = await self.state_manager.get_modified_files()
if self.state_manager.workspace_is_empty():
# NOTE: this will currently get triggered on a new project, but will do
# nothing as there's no files in the database.
log.info("Detected empty workspace, restoring state from the database.")
await self.state_manager.restore_files()
elif modified_files:
await self.send_message(f"We found {len(modified_files)} new and/or modified files.")
await self.ui.send_modified_files(modified_files)
hint = "".join(
[
"If you would like Pythagora to import those changes, click 'Yes'.\n",
"Clicking 'No' means Pythagora will restore (overwrite) all files to the last stored state.\n",
]
)
use_changes = await self.ask_question(
question="Would you like to keep your changes?",
buttons={
"yes": "Yes, keep my changes",
"no": "No, restore last Pythagora state",
},
buttons_only=True,
hint=hint,
)
if use_changes.button == "yes":
log.debug("Importing offline changes into Pythagora.")
await self.import_files()
else:
log.debug("Restoring last stored state.")
await self.state_manager.restore_files()
log.info("Offline changes check done.")
except UserInterruptError:
if self.state_manager.workspace_is_empty():
# NOTE: this will currently get triggered on a new project, but will do
# nothing as there's no files in the database.
log.info("Detected empty workspace, restoring state from the database.")
await self.state_manager.restore_files()
log.debug("User interrupted the offline changes check, restoring files.")
return
elif modified_files:
await self.send_message(f"We found {len(modified_files)} new and/or modified files.")
hint = "".join(
[
"If you would like Pythagora to import those changes, click 'Yes'.\n",
"Clicking 'No' means Pythagora will restore (overwrite) all files to the last stored state.\n",
]
)
use_changes = await self.ask_question(
question="Would you like to keep your changes?",
buttons={
"yes": "Yes, keep my changes",
"no": "No, restore last Pythagora state",
},
buttons_only=True,
hint=hint,
)
if use_changes.button == "yes":
log.debug("Importing offline changes into Pythagora.")
await self.import_files()
else:
log.debug("Restoring last stored state.")
await self.state_manager.restore_files()
log.info("Offline changes check done.")
async def handle_done(self, agent: BaseAgent, response: AgentResponse) -> AgentResponse:
"""
@ -424,25 +129,23 @@ class Orchestrator(BaseAgent, GitMixin):
will trigger the HumanInput agent to ask the user to provide the required input.
"""
if self.next_state and self.next_state.tasks:
n_epics = len(self.next_state.epics)
n_finished_epics = n_epics - len(self.next_state.unfinished_epics)
n_tasks = len(self.next_state.tasks)
n_finished_tasks = n_tasks - len(self.next_state.unfinished_tasks)
n_iterations = len(self.next_state.iterations)
n_finished_iterations = n_iterations - len(self.next_state.unfinished_iterations)
n_steps = len(self.next_state.steps)
n_finished_steps = n_steps - len(self.next_state.unfinished_steps)
log.debug(
f"Agent {agent.__class__.__name__} is done, "
f"committing state for step {self.current_state.step_index}: "
f"{n_finished_epics}/{n_epics} epics, "
f"{n_finished_tasks}/{n_tasks} tasks, "
f"{n_finished_iterations}/{n_iterations} iterations, "
f"{n_finished_steps}/{n_steps} dev steps."
)
n_epics = len(self.next_state.epics)
n_finished_epics = n_epics - len(self.next_state.unfinished_epics)
n_tasks = len(self.next_state.tasks)
n_finished_tasks = n_tasks - len(self.next_state.unfinished_tasks)
n_iterations = len(self.next_state.iterations)
n_finished_iterations = n_iterations - len(self.next_state.unfinished_iterations)
n_steps = len(self.next_state.steps)
n_finished_steps = n_steps - len(self.next_state.unfinished_steps)
log.debug(
f"Agent {agent.__class__.__name__} is done, "
f"committing state for step {self.current_state.step_index}: "
f"{n_finished_epics}/{n_epics} epics, "
f"{n_finished_tasks}/{n_tasks} tasks, "
f"{n_finished_iterations}/{n_iterations} iterations, "
f"{n_finished_steps}/{n_steps} dev steps."
)
await self.state_manager.commit()
# If there are any new or modified files changed outside Pythagora,
@ -451,48 +154,54 @@ class Orchestrator(BaseAgent, GitMixin):
import_files_response = await self.import_files()
# If any of the files are missing metadata/descriptions, those need to be filled-in
missing_descriptions = [
file.path for file in self.current_state.files if not file.content.meta.get("description")
]
missing_descriptions = [file.path for file in self.current_state.files if not file.meta.get("description")]
if missing_descriptions:
log.debug(f"Some files are missing descriptions: {', '.join(missing_descriptions)}, requesting analysis")
return AgentResponse.describe_files(self)
return import_files_response
def create_agent(self, prev_response: Optional[AgentResponse]) -> Union[List[BaseAgent], BaseAgent]:
def create_agent(self, prev_response: Optional[AgentResponse]) -> BaseAgent:
state = self.current_state
if prev_response:
if prev_response.type in [ResponseType.CANCEL, ResponseType.ERROR]:
return ErrorHandler(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.CODE_REVIEW:
return CodeReviewer(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.CODE_REVIEW_FEEDBACK:
return CodeMonkey(self.state_manager, self.ui, prev_response=prev_response, step=state.current_step)
if prev_response.type == ResponseType.DESCRIBE_FILES:
return CodeMonkey(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.INPUT_REQUIRED:
# FIXME: HumanInput should be on the whole time and intercept chat/interrupt
return HumanInput(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.TASK_REVIEW_FEEDBACK:
return Developer(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.IMPORT_PROJECT:
return Importer(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.EXTERNAL_DOCS_REQUIRED:
return ExternalDocumentation(self.state_manager, self.ui, prev_response=prev_response)
if prev_response.type == ResponseType.UPDATE_SPECIFICATION:
return SpecWriter(self.state_manager, self.ui, prev_response=prev_response, args=self.args)
if not state.epics:
return Wizard(self.state_manager, self.ui, process_manager=self.process_manager)
elif state.epics and not state.epics[0].get("description"):
# New project: ask the Spec Writer to refine and save the project specification
return SpecWriter(self.state_manager, self.ui, process_manager=self.process_manager, args=self.args)
elif state.current_epic and state.current_epic.get("source") == "frontend":
# Build frontend
return Frontend(self.state_manager, self.ui, process_manager=self.process_manager)
if not state.specification.description:
if state.files:
# The project has been imported, but not analyzed yet
return Importer(self.state_manager, self.ui)
else:
# New project: ask the Spec Writer to refine and save the project specification
return SpecWriter(self.state_manager, self.ui, process_manager=self.process_manager)
elif not state.specification.architecture:
# Ask the Architect to design the project architecture and determine dependencies
return Architect(self.state_manager, self.ui, process_manager=self.process_manager)
elif not self.current_state.unfinished_tasks or (state.specification.templates and not state.files):
# Ask the Tech Lead to break down the initial project or feature into tasks and apply project templates
elif (
not state.epics
or not self.current_state.unfinished_tasks
or (state.specification.template and not state.files)
):
# Ask the Tech Lead to break down the initial project or feature into tasks and apply project template
return TechLead(self.state_manager, self.ui, process_manager=self.process_manager)
if state.current_task and state.docs is None and state.specification.complexity != Complexity.SIMPLE:
return ExternalDocumentation(self.state_manager, self.ui)
# Current task status must be checked before Developer is called because we might want
# to skip it instead of breaking it down
current_task_status = state.current_task.get("status") if state.current_task else None
@ -502,9 +211,12 @@ class Orchestrator(BaseAgent, GitMixin):
if current_task_status == TaskStatus.REVIEWED:
# User reviewed the task, call TechnicalWriter to see if documentation needs to be updated
return TechnicalWriter(self.state_manager, self.ui)
elif current_task_status in [TaskStatus.DOCUMENTED, TaskStatus.SKIPPED]:
elif current_task_status == TaskStatus.DOCUMENTED:
# After documentation is done, call TechLead update the development plan (remaining tasks)
return TechLead(self.state_manager, self.ui)
elif current_task_status in [TaskStatus.EPIC_UPDATED, TaskStatus.SKIPPED]:
# Task is fully done or skipped, call TaskCompleter to mark it as completed
return TaskCompleter(self.state_manager, self.ui, process_manager=self.process_manager)
return TaskCompleter(self.state_manager, self.ui)
if not state.steps and not state.iterations:
# Ask the Developer to break down current task into actionable steps
@ -516,59 +228,29 @@ class Orchestrator(BaseAgent, GitMixin):
return self.create_agent_for_step(state.current_step)
if state.unfinished_iterations:
current_iteration_status = state.current_iteration["status"]
if current_iteration_status == IterationStatus.HUNTING_FOR_BUG:
# Triggering the bug hunter to start the hunt
return BugHunter(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.START_PAIR_PROGRAMMING:
# Pythagora cannot solve the issue so we're starting pair programming
return BugHunter(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.AWAITING_LOGGING:
# Get the developer to implement logs needed for debugging
return Developer(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.AWAITING_BUG_FIX:
# Get the developer to implement the bug fix for debugging
if state.current_iteration["description"]:
# Break down the next iteration into steps
return Developer(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.IMPLEMENT_SOLUTION:
# Get the developer to implement the "change" requested by the user
return Developer(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.AWAITING_USER_TEST:
# Getting the bug hunter to ask the human to test the bug fix
return BugHunter(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.AWAITING_BUG_REPRODUCTION:
# Getting the bug hunter to ask the human to reproduce the bug
return BugHunter(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.FIND_SOLUTION:
# Find solution to the iteration problem
return Troubleshooter(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.PROBLEM_SOLVER:
# Call Problem Solver if the user said "I'm stuck in a loop"
else:
# We need to iterate over the current task but there's no solution, as Pythagora
# is stuck in a loop, and ProblemSolver needs to find alternative solutions.
return ProblemSolver(self.state_manager, self.ui)
elif current_iteration_status == IterationStatus.NEW_FEATURE_REQUESTED:
# Call Spec Writer to add the "change" requested by the user to project specification
return SpecWriter(self.state_manager, self.ui, args=self.args)
# We have just finished the task, call Troubleshooter to ask the user to review
return Troubleshooter(self.state_manager, self.ui)
def create_agent_for_step(self, step: dict) -> Union[List[BaseAgent], BaseAgent]:
def create_agent_for_step(self, step: dict) -> BaseAgent:
step_type = step.get("type")
if step_type == "save_file":
steps = self.current_state.get_steps_of_type("save_file")
parallel = []
for step in steps:
parallel.append(CodeMonkey(self.state_manager, self.ui, step=step))
return parallel
return CodeMonkey(self.state_manager, self.ui, step=step)
elif step_type == "command":
return self.executor.for_step(step)
elif step_type == "human_intervention":
return HumanInput(self.state_manager, self.ui, step=step)
elif step_type == "review_task":
return LegacyHandler(self.state_manager, self.ui, data={"type": "review_task"})
return TaskReviewer(self.state_manager, self.ui)
elif step_type == "create_readme":
return TechnicalWriter(self.state_manager, self.ui)
elif step_type == "utility_function":
return Developer(self.state_manager, self.ui)
else:
raise ValueError(f"Unknown step type: {step_type}")
@ -584,7 +266,7 @@ class Orchestrator(BaseAgent, GitMixin):
input_required_files: list[dict[str, int]] = []
for file in imported_files:
for line in self.state_manager.get_input_required(file.content.content, file.path):
for line in self.state_manager.get_input_required(file.content.content):
input_required_files.append({"file": file.path, "line": line})
if input_required_files:
@ -598,24 +280,22 @@ class Orchestrator(BaseAgent, GitMixin):
return None
async def init_ui(self):
project_details = self.state_manager.get_project_info()
await self.ui.send_project_info(
project_details["name"], project_details["id"], project_details["folderName"], project_details["createdAt"]
)
await self.ui.send_project_root(self.state_manager.get_full_project_root())
await self.ui.loading_finished()
if self.current_state.epics:
if len(self.current_state.epics) > 3:
await self.ui.send_project_stage(ProjectStage.CODING)
if len(self.current_state.epics) > 2:
# We only want to send previous features, ie. exclude current one and the initial project (first epic)
await self.ui.send_features_list([e["description"] for e in self.current_state.epics[2:-1]])
await self.ui.send_features_list([e["description"] for e in self.current_state.epics[1:-1]])
elif self.current_state.specification.description:
await self.ui.send_project_stage(ProjectStage.ARCHITECTURE)
else:
await self.ui.send_project_stage(ProjectStage.DESCRIPTION)
if self.current_state.specification.description:
await self.ui.send_project_description(
{
"project_description": self.current_state.specification.description,
"project_type": self.current_state.branch.project.project_type,
}
)
await self.ui.send_project_description(self.current_state.specification.description)
async def update_stats(self):
if self.current_state.steps and self.current_state.current_step:

@ -6,7 +6,6 @@ from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.agents.troubleshooter import IterationPromptMixin
from core.db.models.project_state import IterationStatus
from core.llm.parser import JSONParser
from core.log import get_logger
@ -23,7 +22,6 @@ class AlternativeSolutions(BaseModel):
)
# TODO: add next state actions whenever this agent is reactivated
class ProblemSolver(IterationPromptMixin, BaseAgent):
agent_type = "problem-solver"
display_name = "Problem Solver"
@ -47,7 +45,7 @@ class ProblemSolver(IterationPromptMixin, BaseAgent):
return await self.try_alternative_solutions()
async def generate_alternative_solutions(self):
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
@ -100,7 +98,6 @@ class ProblemSolver(IterationPromptMixin, BaseAgent):
self.next_state_iteration["alternative_solutions"][index]["tried"] = True
self.next_state_iteration["description"] = llm_solution
self.next_state_iteration["attempts"] = self.iteration["attempts"] + 1
self.next_state_iteration["status"] = IterationStatus.PROBLEM_SOLVER
self.next_state.flag_iterations_as_modified()
return AgentResponse.done(self)

@ -24,24 +24,24 @@ class ResponseType(str, Enum):
EXIT = "exit"
"""Pythagora should exit."""
CODE_REVIEW = "code-review"
"""Agent is requesting a review of the created code."""
CODE_REVIEW_FEEDBACK = "code-review-feedback"
"""Agent is providing feedback on the code review."""
DESCRIBE_FILES = "describe-files"
"""Analysis of the files in the project is requested."""
INPUT_REQUIRED = "input-required"
"""User needs to modify a line in the generated code."""
TASK_REVIEW_FEEDBACK = "task-review-feedback"
"""Agent is providing feedback on the entire task."""
IMPORT_PROJECT = "import-project"
"""User wants to import an existing project."""
EXTERNAL_DOCS_REQUIRED = "external-docs-required"
"""We need to fetch external docs for a task."""
UPDATE_SPECIFICATION = "update-specification"
"""We need to update the project specification."""
CREATE_SPECIFICATION = "create-specification"
"""We need to create the project specification."""
class AgentResponse:
type: ResponseType = ResponseType.DONE
@ -77,34 +77,63 @@ class AgentResponse:
return AgentResponse(type=ResponseType.EXIT, agent=agent)
@staticmethod
def describe_files(agent: "BaseAgent") -> "AgentResponse":
return AgentResponse(type=ResponseType.DESCRIBE_FILES, agent=agent)
def code_review(
agent: "BaseAgent",
path: str,
instructions: str,
old_content: str,
new_content: str,
attempt: int,
) -> "AgentResponse":
return AgentResponse(
type=ResponseType.CODE_REVIEW,
agent=agent,
data={
"path": path,
"instructions": instructions,
"old_content": old_content,
"new_content": new_content,
"attempt": attempt,
},
)
@staticmethod
def input_required(agent: "BaseAgent", files: list[dict[str, int]]) -> "AgentResponse":
return AgentResponse(type=ResponseType.INPUT_REQUIRED, agent=agent, data={"files": files})
def code_review_feedback(
agent: "BaseAgent",
new_content: str,
approved_content: str,
feedback: str,
attempt: int,
) -> "AgentResponse":
return AgentResponse(
type=ResponseType.CODE_REVIEW_FEEDBACK,
agent=agent,
data={
"new_content": new_content,
"approved_content": approved_content,
"feedback": feedback,
"attempt": attempt,
},
)
@staticmethod
def import_project(agent: "BaseAgent") -> "AgentResponse":
return AgentResponse(type=ResponseType.IMPORT_PROJECT, agent=agent)
def describe_files(agent: "BaseAgent") -> "AgentResponse":
return AgentResponse(type=ResponseType.DESCRIBE_FILES, agent=agent)
@staticmethod
def external_docs_required(agent: "BaseAgent") -> "AgentResponse":
return AgentResponse(type=ResponseType.EXTERNAL_DOCS_REQUIRED, agent=agent)
def input_required(agent: "BaseAgent", files: list[dict[str, int]]) -> "AgentResponse":
return AgentResponse(type=ResponseType.INPUT_REQUIRED, agent=agent, data={"files": files})
@staticmethod
def update_specification(agent: "BaseAgent", description: str) -> "AgentResponse":
def task_review_feedback(agent: "BaseAgent", feedback: str) -> "AgentResponse":
return AgentResponse(
type=ResponseType.UPDATE_SPECIFICATION,
type=ResponseType.TASK_REVIEW_FEEDBACK,
agent=agent,
data={
"description": description,
"feedback": feedback,
},
)
@staticmethod
def create_specification(agent: "BaseAgent") -> "AgentResponse":
return AgentResponse(
type=ResponseType.CREATE_SPECIFICATION,
agent=agent,
)
def import_project(agent: "BaseAgent") -> "AgentResponse":
return AgentResponse(type=ResponseType.IMPORT_PROJECT, agent=agent)

@ -1,17 +1,22 @@
import secrets
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse, ResponseType
from core.config import DEFAULT_AGENT_NAME, SPEC_WRITER_AGENT_NAME
from core.config.actions import SPEC_CHANGE_FEATURE_STEP_NAME, SPEC_CHANGE_STEP_NAME, SPEC_CREATE_STEP_NAME
from core.agents.response import AgentResponse
from core.db.models import Complexity
from core.db.models.project_state import IterationStatus
from core.llm.parser import StringParser
from core.log import get_logger
from core.telemetry import telemetry
from core.templates.registry import PROJECT_TEMPLATES
from core.ui.base import ProjectStage
from core.templates.example_project import (
DEFAULT_EXAMPLE_PROJECT,
EXAMPLE_PROJECTS,
)
# If the project description is less than this, perform an analysis using LLM
ANALYZE_THRESHOLD = 1500
# URL to the wiki page with tips on how to write a good project description
INITIAL_PROJECT_HOWTO_URL = (
"https://github.com/Pythagora-io/gpt-pilot/wiki/How-to-write-a-good-initial-project-description"
)
SPEC_STEP_NAME = "Create specification"
log = get_logger(__name__)
@ -21,336 +26,140 @@ class SpecWriter(BaseAgent):
display_name = "Spec Writer"
async def run(self) -> AgentResponse:
current_iteration = self.current_state.current_iteration
if current_iteration is not None and current_iteration.get("status") == IterationStatus.NEW_FEATURE_REQUESTED:
return await self.update_spec(iteration_mode=True)
elif self.prev_response and self.prev_response.type == ResponseType.UPDATE_SPECIFICATION:
return await self.update_spec(iteration_mode=False)
elif not self.current_state.specification.description:
return await self.initialize_spec_and_project()
else:
return await self.change_spec()
async def apply_template(self):
"""
Applies a template to the frontend.
"""
options = self.current_state.knowledge_base.user_options
if options["auth_type"] == "api_key" or options["auth_type"] == "none":
template_name = "vite_react_swagger"
else:
template_name = "vite_react"
template_class = PROJECT_TEMPLATES.get(template_name)
if not template_class:
log.error(f"Project template not found: {template_name}")
return
template = template_class(
options,
self.state_manager,
self.process_manager,
)
if not self.state_manager.template:
self.state_manager.template = {}
self.state_manager.template["template"] = template
log.info(f"Applying project template: {template.name}")
summary = await template.apply()
self.next_state.relevant_files = template.relevant_files
self.next_state.modified_files = {}
self.next_state.specification.template_summary = summary
async def initialize_spec_and_project(self) -> AgentResponse:
self.next_state.action = SPEC_CREATE_STEP_NAME
await self.ui.send_project_stage({"stage": ProjectStage.PROJECT_DESCRIPTION})
await self.ui.clear_main_logs()
# Check if initial_prompt is provided in command line arguments
if self.args and self.args.initial_prompt:
description = self.args.initial_prompt.strip()
await self.ui.send_back_logs(
[
{
"title": "",
"project_state_id": "spec",
"labels": [""],
"convo": [
{"role": "assistant", "content": "Please describe the app you want to build."},
{"role": "user", "content": description},
],
}
]
)
else:
user_description = await self.ask_question(
"Please describe the app you want to build.",
allow_empty=False,
full_screen=True,
verbose=True,
extra_info={
"chat_section_tip": "\"Some text <a href='https://example.com'>link text</a> on how to build apps with Pythagora.\""
},
)
description = user_description.text.strip()
await self.ui.send_back_logs(
[
{
"title": "",
"project_state_id": self.current_state.id,
"labels": [""],
"convo": [
{"role": "assistant", "content": "Please describe the app you want to build."},
{"role": "user", "content": description},
],
}
]
)
await self.ui.send_back_logs(
[
{
"title": "Writing Specification",
"project_state_id": "spec",
"labels": ["E1 / T1", "Specs", "working"],
"disallow_reload": True,
}
]
)
await self.ui.send_front_logs_headers("specs_0", ["E1 / T1", "Writing Specification", "working"], "")
await self.send_message(
"## Write specification\n\nPythagora is generating a detailed specification for app based on your input.",
# project_state_id="setup",
)
llm = self.get_llm(SPEC_WRITER_AGENT_NAME, stream_output=True, route="forwardToCenter")
convo = AgentConvo(self).template(
"build_full_specification",
initial_prompt=description,
)
llm_assisted_description = await llm(convo)
await self.ui.send_project_stage({"stage": ProjectStage.PROJECT_NAME})
llm = self.get_llm(DEFAULT_AGENT_NAME)
convo = AgentConvo(self).template(
"project_name",
description=llm_assisted_description,
)
llm_response: str = await llm(convo, temperature=0)
project_name = llm_response.strip()
self.state_manager.project.name = project_name
self.state_manager.project.folder_name = project_name.replace(" ", "_").replace("-", "_")
self.state_manager.file_system = await self.state_manager.init_file_system(load_existing=False)
self.process_manager.root_dir = self.state_manager.file_system.root
self.next_state.knowledge_base.user_options["original_description"] = description
self.next_state.knowledge_base.user_options["project_description"] = llm_assisted_description
self.next_state.specification = self.current_state.specification.clone()
self.next_state.specification.description = llm_assisted_description
self.next_state.specification.original_description = description
return AgentResponse.done(self)
async def change_spec(self) -> AgentResponse:
llm = self.get_llm(SPEC_WRITER_AGENT_NAME, stream_output=True, route="forwardToCenter")
description = self.current_state.specification.original_description
current_description = self.current_state.specification.description
convo = AgentConvo(self).template(
"build_full_specification",
initial_prompt=self.current_state.specification.description.strip(),
response = await self.ask_question(
"Describe your app in as much detail as possible",
allow_empty=False,
buttons={
# FIXME: must be lowercase becase VSCode doesn't recognize it otherwise. Needs a fix in the extension
"continue": "continue",
"example": "Start an example project",
"import": "Import an existing project",
},
)
if response.cancelled:
return AgentResponse.error(self, "No project description")
while True:
user_done_with_description = await self.ask_question(
"Are you satisfied with the project description?",
buttons={
"yes": "Yes",
"no": "No, I want to add more details",
},
default="yes",
buttons_only=True,
)
if user_done_with_description.button == "yes":
await self.ui.send_project_stage({"stage": ProjectStage.SPECS_FINISHED})
break
elif user_done_with_description.button == "no":
await self.send_message("## What would you like to add?")
user_add_to_spec = await self.ask_question(
"What would you like to add?",
allow_empty=False,
)
else:
user_add_to_spec = user_done_with_description
await self.send_message("## Refining specification\n\nPythagora is refining the specs based on your input.")
# if user edits the spec with extension, it will be commited to db immediately, so we have to check if the description has changed
if current_description != self.current_state.specification.description:
convo = AgentConvo(self).template(
"build_full_specification",
initial_prompt=self.current_state.specification.description.strip(),
)
convo = convo.template("add_to_specification", user_message=user_add_to_spec.text.strip())
if response.button == "import":
return AgentResponse.import_project(self)
if len(convo.messages) > 6:
convo.slice(1, 4)
if response.button == "example":
await self.prepare_example_project(DEFAULT_EXAMPLE_PROJECT)
return AgentResponse.done(self)
# await self.ui.set_important_stream()
llm_assisted_description = await llm(convo)
elif response.button == "continue":
# FIXME: Workaround for the fact that VSCode "continue" button does
# nothing but repeat the question. We reproduce this bug for bug here.
return AgentResponse.done(self)
# when llm generates a new spec - make it the new default spec, even if user edited it before - because it will be shown in the extension
self.current_state.specification.description = llm_assisted_description
convo = convo.assistant(llm_assisted_description)
await self.ui.clear_main_logs()
await self.ui.send_back_logs(
[
{
"title": "Writing Specification",
"project_state_id": "spec", # self.current_state.id,
"labels": ["E1 / T1", "Specs", "done"],
"convo": [
{
"role": "assistant",
"content": "What do you want to build?",
},
{
"role": "user",
"content": self.current_state.specification.original_description,
},
],
"disallow_reload": True,
}
]
)
spec = response.text.strip()
llm = self.get_llm(SPEC_WRITER_AGENT_NAME)
convo = AgentConvo(self).template(
"need_auth",
description=self.current_state.specification.description,
complexity = await self.check_prompt_complexity(spec)
await telemetry.trace_code_event(
"project-description",
{
"initial_prompt": spec,
"complexity": complexity,
},
)
llm_response: str = await llm(convo, temperature=0)
auth = llm_response.strip().lower() == "yes"
if auth:
self.next_state.knowledge_base.user_options["auth"] = auth
self.next_state.knowledge_base.user_options["jwt_secret"] = secrets.token_hex(32)
self.next_state.knowledge_base.user_options["refresh_token_secret"] = secrets.token_hex(32)
self.next_state.flag_knowledge_base_as_modified()
# if we reload the project from the 1st project state, state_manager.template will be None
if self.state_manager.template:
self.state_manager.template["description"] = self.current_state.specification.description
else:
# if we do not set this and reload the project, we will load the "old" project description we entered before reload
self.next_state.epics[0]["description"] = self.current_state.specification.description
if len(spec) < ANALYZE_THRESHOLD and complexity != Complexity.SIMPLE:
spec = await self.analyze_spec(spec)
spec = await self.review_spec(spec)
self.next_state.specification = self.current_state.specification.clone()
self.next_state.specification.original_description = description
self.next_state.specification.description = self.current_state.specification.description
complexity = await self.check_prompt_complexity(self.current_state.specification.description)
self.next_state.specification.description = spec
self.next_state.specification.complexity = complexity
telemetry.set("initial_prompt", description)
telemetry.set("updated_prompt", self.current_state.specification.description)
telemetry.set("initial_prompt", spec)
telemetry.set("is_complex_app", complexity != Complexity.SIMPLE)
await self.ui.send_project_description(
{
"project_description": self.current_state.specification.description,
"project_type": self.current_state.branch.project.project_type,
}
)
self.next_state.action = SPEC_STEP_NAME
return AgentResponse.done(self)
await telemetry.trace_code_event(
"project-description",
{
"complexity": complexity,
"initial_prompt": description,
"llm_assisted_prompt": self.current_state.specification.description,
},
)
async def check_prompt_complexity(self, prompt: str) -> str:
await self.send_message("Checking the complexity of the prompt ...")
llm = self.get_llm()
convo = AgentConvo(self).template("prompt_complexity", prompt=prompt)
llm_response: str = await llm(convo, temperature=0, parser=StringParser())
return llm_response.lower()
self.next_state.epics = [
{
"id": self.current_state.epics[0]["id"],
"name": "Build frontend",
"source": "frontend",
"description": self.current_state.specification.description,
"messages": [],
"summary": None,
"completed": False,
}
]
async def prepare_example_project(self, example_name: str):
example_description = EXAMPLE_PROJECTS[example_name]["description"].strip()
if not self.state_manager.async_tasks:
self.state_manager.async_tasks = []
await self.apply_template()
log.debug(f"Starting example project: {example_name}")
await self.send_message(f"Starting example project with description:\n\n{example_description}")
return AgentResponse.done(self)
spec = self.current_state.specification.clone()
spec.example_project = example_name
spec.description = example_description
spec.complexity = EXAMPLE_PROJECTS[example_name]["complexity"]
self.next_state.specification = spec
async def update_spec(self, iteration_mode) -> AgentResponse:
if iteration_mode:
self.next_state.action = SPEC_CHANGE_FEATURE_STEP_NAME
feature_description = self.current_state.current_iteration["user_feedback"]
else:
self.next_state.action = SPEC_CHANGE_STEP_NAME
feature_description = self.prev_response.data["description"]
telemetry.set("initial_prompt", spec.description)
telemetry.set("example_project", example_name)
telemetry.set("is_complex_app", spec.complexity != Complexity.SIMPLE)
await self.send_message(
f"Making the following changes to project specification:\n\n{feature_description}\n\nUpdated project specification:"
async def analyze_spec(self, spec: str) -> str:
msg = (
"Your project description seems a bit short. "
"The better you can describe the project, the better GPT Pilot will understand what you'd like to build.\n\n"
f"Here are some tips on how to better describe the project: {INITIAL_PROJECT_HOWTO_URL}\n\n"
"Let's start by refining your project idea:"
)
llm = self.get_llm(SPEC_WRITER_AGENT_NAME, stream_output=True, route="forwardToCenter")
convo = AgentConvo(self).template("add_new_feature", feature_description=feature_description)
llm_response: str = await llm(convo, temperature=0, parser=StringParser())
updated_spec = llm_response.strip()
await self.ui.generate_diff(
"project_specification", self.current_state.specification.description, updated_spec, source=self.ui_source
)
user_response = await self.ask_question(
"Do you accept these changes to the project specification?",
buttons={"yes": "Yes", "no": "No"},
default="yes",
buttons_only=True,
)
await self.ui.close_diff()
await self.send_message(msg)
if user_response.button == "yes":
self.next_state.specification = self.current_state.specification.clone()
self.next_state.specification.description = updated_spec
telemetry.set("updated_prompt", updated_spec)
llm = self.get_llm()
convo = AgentConvo(self).template("ask_questions").user(spec)
n_questions = 0
n_answers = 0
if iteration_mode:
self.next_state.current_iteration["status"] = IterationStatus.FIND_SOLUTION
self.next_state.flag_iterations_as_modified()
else:
complexity = await self.check_prompt_complexity(feature_description)
self.next_state.current_epic["complexity"] = complexity
while True:
response: str = await llm(convo)
if len(response) > 500:
# The response is too long for it to be a question, assume it's the spec
confirm = await self.ask_question(
(
"Can we proceed with this project description? If so, just press ENTER. "
"Otherwise, please tell me what's missing or what you'd like to add."
),
allow_empty=True,
buttons={"continue": "continue"},
)
if confirm.cancelled or confirm.button == "continue" or confirm.text == "":
await telemetry.trace_code_event(
"spec-writer-questions",
{
"num_questions": n_questions,
"num_answers": n_answers,
"new_spec": spec,
},
)
return spec
convo.user(confirm.text)
return AgentResponse.done(self)
else:
convo.assistant(response)
async def check_prompt_complexity(self, prompt: str) -> str:
is_feature = self.current_state.epics and len(self.current_state.epics) > 2
await self.send_message("Checking the complexity of the prompt...\n")
llm = self.get_llm(SPEC_WRITER_AGENT_NAME)
convo = AgentConvo(self).template(
"prompt_complexity",
prompt=prompt,
is_feature=is_feature,
)
llm_response: str = await llm(convo, temperature=0, parser=StringParser())
log.info(f"Complexity check response: {llm_response}")
return llm_response.lower()
n_questions += 1
user_response = await self.ask_question(
response,
buttons={"skip": "Skip questions"},
)
if user_response.cancelled or user_response.button == "skip":
convo.user(
"This is enough clarification, you have all the information. "
"Please output the spec now, without additional comments or questions."
)
response: str = await llm(convo)
return response
n_answers += 1
convo.user(user_response.text)
async def review_spec(self, spec: str) -> str:
convo = AgentConvo(self).template("review_spec", spec=spec)
llm = self.get_llm()
llm_response: str = await llm(convo, temperature=0)
additional_info = llm_response.strip()
if additional_info:
spec += "\nAdditional info/examples:\n" + additional_info
return spec

@ -1,23 +1,18 @@
from core.agents.base import BaseAgent
from core.agents.git import GitMixin
from core.agents.response import AgentResponse
from core.config.actions import TC_TASK_DONE
from core.log import get_logger
from core.telemetry import telemetry
log = get_logger(__name__)
class TaskCompleter(BaseAgent, GitMixin):
class TaskCompleter(BaseAgent):
agent_type = "pythagora"
display_name = "Pythagora"
async def run(self) -> AgentResponse:
if self.state_manager.git_available and self.state_manager.git_used:
await self.git_commit()
current_task_index1 = self.current_state.tasks.index(self.current_state.current_task) + 1
self.next_state.action = TC_TASK_DONE.format(current_task_index1)
self.next_state.action = f"Task #{current_task_index1} complete"
self.next_state.complete_task()
await self.state_manager.log_task_completed()
tasks = self.current_state.tasks
@ -43,16 +38,8 @@ class TaskCompleter(BaseAgent, GitMixin):
if current_task_index1 == len(tasks):
if source == "app":
await self.ui.send_app_finished(
app_id=str(self.state_manager.project.id),
app_name=self.state_manager.project.name,
folder_name=self.state_manager.project.folder_name,
)
await self.ui.send_app_finished()
elif source == "feature":
await self.ui.send_feature_finished(
app_id=str(self.state_manager.project.id),
app_name=self.state_manager.project.name,
folder_name=self.state_manager.project.folder_name,
)
await self.ui.send_feature_finished()
return AgentResponse.done(self)

@ -0,0 +1,53 @@
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.log import get_logger
log = get_logger(__name__)
class TaskReviewer(BaseAgent):
agent_type = "task-reviewer"
display_name = "Task Reviewer"
async def run(self) -> AgentResponse:
response = await self.review_code_changes()
self.next_state.complete_step()
return response
async def review_code_changes(self) -> AgentResponse:
"""
Review all the code changes during current task.
"""
log.debug(f"Reviewing code changes for task {self.current_state.current_task['description']}")
await self.send_message("Reviewing the task implementation ...")
all_feedbacks = [
iteration["user_feedback"].replace("```", "").strip()
for iteration in self.current_state.iterations
# Some iterations are created by the task reviewer and have no user feedback
if iteration["user_feedback"]
]
files_before_modification = self.current_state.modified_files
files_after_modification = [
(file.path, file.content.content)
for file in self.current_state.files
if (file.path in files_before_modification)
]
llm = self.get_llm()
# TODO instead of sending files before and after maybe add nice way to show diff for multiple files
convo = AgentConvo(self).template(
"review_task",
current_task=self.current_state.current_task,
all_feedbacks=all_feedbacks,
files_before_modification=files_before_modification,
files_after_modification=files_after_modification,
)
llm_response: str = await llm(convo, temperature=0.7)
if llm_response.strip().lower() == "done":
return AgentResponse.done(self)
else:
return AgentResponse.task_review_feedback(self, llm_response)

@ -1,363 +1,160 @@
import asyncio
from typing import Optional
from uuid import uuid4
from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.mixins import RelevantFilesMixin
from core.agents.response import AgentResponse
from core.config import TECH_LEAD_EPIC_BREAKDOWN, TECH_LEAD_PLANNING
from core.config.actions import (
TL_CREATE_INITIAL_EPIC,
TL_CREATE_PLAN,
TL_INITIAL_PROJECT_NAME,
TL_START_FEATURE,
)
from core.db.models import Complexity
from core.db.models.project_state import TaskStatus
from core.llm.parser import JSONParser
from core.log import get_logger
from core.telemetry import telemetry
from core.templates.registry import PROJECT_TEMPLATES
from core.ui.base import ProjectStage, pythagora_source, success_source
from core.utils.text import trim_logs
from core.templates.example_project import EXAMPLE_PROJECTS
from core.templates.registry import apply_project_template, get_template_description, get_template_summary
from core.ui.base import ProjectStage, success_source
log = get_logger(__name__)
class APIEndpoint(BaseModel):
description: str = Field(description="Description of an API endpoint.")
method: str = Field(description="HTTP method of the API endpoint.")
endpoint: str = Field(description="URL of the API endpoint.")
request_body: dict = Field(description="Request body of the API endpoint.")
response_body: dict = Field(description="Response body of the API endpoint.")
class Epic(BaseModel):
description: str = Field(description="Description of an epic.")
class Task(BaseModel):
description: str = Field(description="Description of a task.")
related_api_endpoints: list[APIEndpoint] = Field(description="API endpoints that will be implemented in this task.")
testing_instructions: str = Field(description="Instructions for testing the task.")
description: str = Field(description=("Very detailed description of a development task."))
class DevelopmentPlan(BaseModel):
plan: list[Epic] = Field(description="List of epics that need to be done to implement the entire plan.")
plan: list[Task] = Field(description="List of development tasks that need to be done to implement the entire plan.")
class EpicPlan(BaseModel):
plan: list[Task] = Field(description="List of tasks that need to be done to implement the entire epic.")
class UpdatedDevelopmentPlan(BaseModel):
updated_current_epic: Task = Field(
description="Updated detailed description of what was implemented while working on the current development task."
)
plan: list[Task] = Field(description="List of unfinished development tasks.")
class TechLead(RelevantFilesMixin, BaseAgent):
class TechLead(BaseAgent):
agent_type = "tech-lead"
display_name = "Tech Lead"
async def run(self) -> AgentResponse:
# Building frontend is the first epic
if len(self.current_state.epics) == 1:
await self.remove_mocked_data()
self.create_initial_project_epic()
current_task_status = self.current_state.current_task.get("status") if self.current_state.current_task else None
if current_task_status and current_task_status == TaskStatus.DOCUMENTED:
return await self.update_epic()
if len(self.current_state.epics) == 0:
if self.current_state.specification.example_project:
self.plan_example_project()
else:
self.create_initial_project_epic()
return AgentResponse.done(self)
# if self.current_state.specification.templates and len(self.current_state.files) < 2:
# await self.apply_project_templates()
# self.next_state.action = "Apply project templates"
# await self.ui.send_epics_and_tasks(
# self.next_state.current_epic["sub_epics"],
# self.next_state.tasks,
# )
#
# inputs = []
# for file in self.next_state.files:
# input_required = self.state_manager.get_input_required(file.content.content)
# if input_required:
# inputs += [{"file": file.path, "line": line} for line in input_required]
#
# if inputs:
# return AgentResponse.input_required(self, inputs)
# else:
# return AgentResponse.done(self)
await self.ui.send_project_stage(ProjectStage.CODING)
if self.current_state.specification.template and not self.current_state.files:
await self.apply_project_template()
self.next_state.action = "Apply project template"
return AgentResponse.done(self)
if self.current_state.current_epic:
await self.remove_mocked_data()
self.next_state.action = "Create a development plan"
return await self.plan_epic(self.current_state.current_epic)
else:
return await self.ask_for_new_feature()
def create_initial_project_epic(self):
self.next_state.action = TL_CREATE_INITIAL_EPIC
log.debug("Creating initial project Epic")
self.next_state.epics = self.current_state.epics + [
log.debug("Creating initial project epic")
self.next_state.epics = [
{
"id": uuid4().hex,
"name": TL_INITIAL_PROJECT_NAME,
"name": "Initial Project",
"source": "app",
"description": self.current_state.specification.description,
"test_instructions": None,
"summary": None,
"completed": False,
"complexity": self.current_state.specification.complexity,
"sub_epics": [],
}
]
self.next_state.relevant_files = None
self.next_state.modified_files = {}
async def apply_project_templates(self):
async def apply_project_template(self) -> Optional[str]:
state = self.current_state
summaries = []
# Only do this for the initial project and if the templates are specified
if len(state.epics) != 1 or not state.specification.templates:
return
for template_name, template_options in state.specification.templates.items():
template_class = PROJECT_TEMPLATES.get(template_name)
if not template_class:
log.error(f"Project template not found: {template_name}")
continue
template = template_class(
template_options,
self.state_manager,
self.process_manager,
)
description = template.description
log.info(f"Applying project template: {template.name}")
await self.send_message(f"Applying project template {description} ...")
summary = await template.apply()
summaries.append(summary)
# Saving template files will fill this in and we want it clear for the first task.
# Only do this for the initial project and if the template is specified
if len(state.epics) != 1 or not state.specification.template:
return None
description = get_template_description(state.specification.template)
log.info(f"Applying project template: {state.specification.template}")
await self.send_message(f"Applying project template {description} ...")
summary = await apply_project_template(
self.current_state.specification.template,
self.state_manager,
self.process_manager,
)
# Saving template files will fill this in and we want it clear for the
# first task.
self.next_state.relevant_files = None
if summaries:
spec = self.current_state.specification.clone()
spec.template_summary = "\n\n".join(summaries)
self.next_state.specification = spec
return summary
async def ask_for_new_feature(self) -> AgentResponse:
if len(self.current_state.epics) > 2:
await self.ui.send_message("Your new feature is complete!", source=success_source)
await self.ui.send_project_stage(
{
"stage": ProjectStage.FEATURE_FINISHED,
"feature_number": len(self.current_state.epics),
}
)
else:
await self.ui.send_message("Your app is DONE! You can start using it right now!", source=success_source)
await self.ui.send_project_stage(
{
"stage": ProjectStage.INITIAL_APP_FINISHED,
}
)
if self.current_state.run_command:
await self.ui.send_run_command(self.current_state.run_command)
await self.ui.send_message("Your app is DONE!!! You can start using it right now!", source=success_source)
log.debug("Asking for new feature")
feature, user_desc = None, None
while True:
response = await self.ask_question(
"Do you want to add a new feature or implement something quickly?",
buttons={
# "feature": "Feature",
"task": "Implement new feature",
# "end": "No, I'm done",
},
buttons_only=True,
)
if response.button == "end" or response.cancelled:
await self.ui.send_message("Thank you for using Pythagora!", source=pythagora_source)
return AgentResponse.exit(self)
if not response.text:
feature = response.button == "feature"
response = await self.ask_question(
"What do you want to implement?",
buttons={"back": "Back"},
allow_empty=False,
)
if response.text:
user_desc = response.text
break
if feature:
await self.ui.send_project_stage(
{
"stage": ProjectStage.STARTING_NEW_FEATURE,
"feature_number": len(self.current_state.epics),
}
)
self.next_state.epics = self.current_state.epics + [
{
"id": uuid4().hex,
"name": f"Feature #{len(self.current_state.epics)}",
"test_instructions": None,
"source": "feature",
"description": user_desc,
"summary": None,
"completed": False,
"complexity": None, # Determined and defined in SpecWriter
"sub_epics": [],
}
]
# Orchestrator will rerun us to break down the new feature epic
self.next_state.action = TL_START_FEATURE.format(len(self.current_state.epics))
return AgentResponse.update_specification(self, user_desc)
else:
# Quick implementation
# TODO send project stage?
# load the previous state, because in this state we have deleted tasks due to epic being completed!
wanted_project_state = await self.state_manager.get_project_state_by_id(self.current_state.prev_state_id)
wanted_project_state.epics[-1]["completed"] = False
self.next_state.epics = wanted_project_state.epics
# Trim logs from existing tasks before adding the new task
if wanted_project_state.tasks:
# Trim logs from all existing tasks
for task in wanted_project_state.tasks:
if task.get("description"):
task["description"] = trim_logs(task["description"])
# Create tasks list with new task (after trimming logs from existing tasks)
self.next_state.tasks = wanted_project_state.tasks + [
{
"id": uuid4().hex,
"description": user_desc,
"instructions": None,
"pre_breakdown_testing_instructions": None,
"status": TaskStatus.TODO,
"sub_epic_id": self.next_state.epics[-1]["sub_epics"][-1]["id"],
"quick_implementation": True,
}
]
# Flag tasks as modified so SQLAlchemy knows to save the changes
self.next_state.flag_epics_as_modified()
self.next_state.flag_tasks_as_modified()
await self.ui.send_epics_and_tasks(
self.next_state.epics[-1].get("sub_epics", []),
self.next_state.tasks,
)
return AgentResponse.done(self)
async def process_epic(self, sub_epic_number, sub_epic):
epic_convo = (
AgentConvo(self)
.template(
"epic_breakdown",
epic_number=sub_epic_number,
epic_description=sub_epic.description,
get_only_api_files=True,
)
.require_schema(EpicPlan)
response = await self.ask_question(
"Do you have a new feature to add to the project? Just write it here",
buttons={"continue": "continue", "end": "No, I'm done"},
allow_empty=True,
)
llm = self.get_llm(TECH_LEAD_EPIC_BREAKDOWN)
epic_plan: EpicPlan = await llm(epic_convo, parser=JSONParser(EpicPlan))
task = {
"id": uuid4().hex,
"description": "",
"instructions": None,
"pre_breakdown_testing_instructions": "",
"status": TaskStatus.TODO,
"sub_epic_id": sub_epic_number,
"related_api_endpoints": [],
}
for epic_task in epic_plan.plan:
task["description"] += (
epic_task.description + " " if epic_task.description.endswith(".") else epic_task.description + ". "
)
task["related_api_endpoints"] += [rae.model_dump() for rae in (epic_task.related_api_endpoints or [])]
task["pre_breakdown_testing_instructions"] += f"{epic_task.description}\n{epic_task.testing_instructions}\n"
if response.cancelled or not response.text:
return AgentResponse.exit(self)
return task
self.next_state.epics = self.current_state.epics + [
{
"id": uuid4().hex,
"name": f"Feature #{len(self.current_state.epics)}",
"test_instructions": None,
"source": "feature",
"description": response.text,
"summary": None,
"completed": False,
"complexity": Complexity.HARD,
}
]
# Orchestrator will rerun us to break down the new feature epic
self.next_state.action = f"Start of feature #{len(self.current_state.epics)}"
return AgentResponse.done(self)
async def plan_epic(self, epic) -> AgentResponse:
self.next_state.action = TL_CREATE_PLAN.format(epic["name"])
log.debug(f"Planning tasks for the epic: {epic['name']}")
await self.send_message("Creating the development plan ...")
if epic.get("source") == "feature":
await self.get_relevant_files_parallel(user_feedback=epic.get("description"))
await self.send_message("Starting to create the action plan for development ...")
llm = self.get_llm(TECH_LEAD_PLANNING)
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
"plan",
epic=epic,
task_type=self.current_state.current_epic.get("source", "app"),
# FIXME: we're injecting summaries to initial description
existing_summary=None,
get_only_api_files=True,
existing_summary=get_template_summary(self.current_state.specification.template),
)
.require_schema(DevelopmentPlan)
)
response: DevelopmentPlan = await llm(convo, parser=JSONParser(DevelopmentPlan))
convo.remove_last_x_messages(1)
await self.send_message("Creating tasks ...")
if epic.get("source") == "feature" or epic.get("complexity") == Complexity.SIMPLE:
self.next_state.current_epic["sub_epics"] = [
{
"id": 1,
"description": epic["name"],
}
]
else:
self.next_state.current_epic["sub_epics"] = [
{
"id": sub_epic_number,
"description": sub_epic.description,
}
for sub_epic_number, sub_epic in enumerate(response.plan, start=1)
]
# Create and gather all epic processing tasks
epic_tasks = []
for sub_epic_number, sub_epic in enumerate(response.plan, start=1):
epic_tasks.append(self.process_epic(sub_epic_number, sub_epic))
all_tasks_results = await asyncio.gather(*epic_tasks)
for tasks_result in all_tasks_results:
self.next_state.tasks.append(tasks_result)
await self.ui.send_epics_and_tasks(
self.next_state.current_epic["sub_epics"],
self.next_state.tasks,
)
await self.update_epics_and_tasks()
await self.ui.send_epics_and_tasks(
self.next_state.current_epic["sub_epics"],
self.next_state.tasks,
)
self.next_state.tasks = self.current_state.tasks + [
{
"id": uuid4().hex,
"description": task.description,
"instructions": None,
"status": TaskStatus.TODO,
}
for task in response.plan
]
await telemetry.trace_code_event(
"development-plan",
{
@ -367,97 +164,66 @@ class TechLead(RelevantFilesMixin, BaseAgent):
)
return AgentResponse.done(self)
# TODO - Move to a separate agent for removing mocked data
async def remove_mocked_data(self):
files = self.current_state.files
for file in files:
file_content = file.content.content
if "pythagora_mocked_data" in file_content:
for line in file_content.split("\n"):
if "pythagora_mocked_data" in line:
file_content = file_content.replace(line + "\n", "")
await self.state_manager.save_file(file.path, file_content)
async def update_epics_and_tasks(self):
if (
self.current_state.current_epic
and self.current_state.current_epic.get("source", "") == "app"
and self.current_state.knowledge_base.user_options.get("auth", False)
):
log.debug("Adding auth task to the beginning of the task list")
self.next_state.tasks.insert(
0,
{
"id": uuid4().hex,
"hardcoded": True,
"description": "Implement and test Login and Register pages",
"instructions": """Open /register page, add your data and click on the "Register" button\nExpected result: You should see a success message in the bottom right corner and you should be redirected to the /login page\n2. On the /login page, add your data and click on the "Login" button\nExpected result: You should see a success message in the bottom right corner and you should be redirected to the home page""",
"test_instructions": """[
{
"title": "Open Register Page",
"action": "Open your web browser and visit 'http://localhost:5173/register'.",
"result": "You should see a success message in the bottom right corner and you should be redirected to the /login page"
},
{
"title": "Open Login Page",
"action": "Open your web browser and visit 'http://localhost:5173/login'.",
"result": "You should see a success message in the bottom right corner and you should be redirected to the home page"
}
]""",
"pre_breakdown_testing_instructions": """Open /register page, add your data and click on the "Register" button\nExpected result: You should see a success message in the bottom right corner and you should be redirected to the /login page\n2. On the /login page, add your data and click on the "Login" button\nExpected result: You should see a success message in the bottom right corner and you should be redirected to the home page""",
"status": TaskStatus.TODO,
"sub_epic_id": 1,
"related_api_endpoints": [
{
"description": "Register a new user",
"method": "POST",
"endpoint": "/api/auth/register",
"request_body": {"email": "string", "password": "string"},
"response_body": {
"id": "integer",
"email": "string",
},
},
{
"description": "Login user",
"method": "POST",
"endpoint": "/api/auth/login",
"request_body": {"username": "string", "password": "string"},
"response_body": {"token": "string"},
},
],
},
async def update_epic(self) -> AgentResponse:
"""
Update the development plan for the current epic.
"""
epic = self.current_state.current_epic
self.next_state.set_current_task_status(TaskStatus.EPIC_UPDATED)
if len(self.next_state.unfinished_tasks) == 1 or not self.current_state.iterations:
# Current task is still "unfinished" at this point, so if it's last task, there's nothing to update
return AgentResponse.done(self)
finished_tasks = [task for task in self.next_state.tasks if task["status"] == TaskStatus.DONE]
finished_tasks.append(self.next_state.current_task)
log.debug(f"Updating development plan for {epic['name']}")
await self.send_message("Updating development plan ...")
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(
"update_plan",
finished_tasks=finished_tasks,
task_type=self.current_state.current_epic.get("source", "app"),
modified_files=[f for f in self.current_state.files if f.path in self.current_state.modified_files],
)
.require_schema(UpdatedDevelopmentPlan)
)
self.next_state.steps = [
{
"completed": True,
"iteration_index": 0,
}
]
self.next_state.flag_tasks_as_modified()
self.next_state.flag_epics_as_modified()
await self.ui.clear_main_logs()
await self.ui.send_project_stage(
response: UpdatedDevelopmentPlan = await llm(
convo,
parser=JSONParser(UpdatedDevelopmentPlan),
temperature=0,
)
log.debug(f"Reworded last task as: {response.updated_current_epic.description}")
finished_tasks[-1]["description"] = response.updated_current_epic.description
self.next_state.tasks = finished_tasks + [
{
"stage": ProjectStage.STARTING_TASK,
"task_index": 0,
"id": uuid4().hex,
"description": task.description,
"instructions": None,
"status": TaskStatus.TODO,
}
)
await self.ui.send_front_logs_headers(
str(self.current_state.id),
["E3 / T1", "Backend", "working"],
self.next_state.tasks[0]["description"],
self.next_state.tasks[0]["id"],
)
for task in response.plan
]
log.debug(f"Updated development plan for {epic['name']}, {len(response.plan)} tasks remaining")
return AgentResponse.done(self)
await self.ui.send_back_logs(
[
{
"title": self.next_state.tasks[0]["description"],
"project_state_id": str(self.next_state.id),
"labels": ["E3 / T1", "Backend", "working"],
}
]
)
def plan_example_project(self):
example_name = self.current_state.specification.example_project
log.debug(f"Planning example project: {example_name}")
example = EXAMPLE_PROJECTS[example_name]
self.next_state.epics = [
{
"name": "Initial Project",
"description": example["description"],
"completed": False,
"complexity": example["complexity"],
}
]
self.next_state.tasks = example["plan"]

@ -1,7 +1,6 @@
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.response import AgentResponse
from core.config.actions import TW_WRITE
from core.db.models.project_state import TaskStatus
from core.log import get_logger
from core.ui.base import success_source
@ -24,7 +23,7 @@ class TechnicalWriter(BaseAgent):
await self.send_congratulations()
await self.create_readme()
self.next_state.action = TW_WRITE
self.next_state.action = "Create README.md"
self.next_state.set_current_task_status(TaskStatus.DOCUMENTED)
return AgentResponse.done(self)
@ -53,7 +52,7 @@ class TechnicalWriter(BaseAgent):
async def create_readme(self):
await self.send_message("Creating README ...")
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
convo = AgentConvo(self).template("create_readme")
llm_response: str = await llm(convo)
await self.state_manager.save_file("README.md", llm_response)

@ -1,4 +1,3 @@
import json
from typing import Optional
from uuid import uuid4
@ -6,16 +5,14 @@ from pydantic import BaseModel, Field
from core.agents.base import BaseAgent
from core.agents.convo import AgentConvo
from core.agents.mixins import ChatWithBreakdownMixin, IterationPromptMixin, RelevantFilesMixin, TestSteps
from core.agents.mixins import IterationPromptMixin
from core.agents.response import AgentResponse
from core.config import TROUBLESHOOTER_GET_RUN_COMMAND
from core.config.actions import TS_ALT_SOLUTION, TS_APP_WORKING, TS_DESCRIBE_ISSUE, TS_TASK_REVIEWED
from core.config import ROUTE_FILES_AGENT_NAME
from core.db.models.file import File
from core.db.models.project_state import IterationStatus, TaskStatus
from core.db.models.project_state import TaskStatus
from core.llm.parser import JSONParser, OptionalCodeBlockParser
from core.log import get_logger
from core.telemetry import telemetry
from core.ui.base import ProjectStage, pythagora_source
log = get_logger(__name__)
@ -32,35 +29,11 @@ class RouteFilePaths(BaseModel):
files: list[str] = Field(description="List of paths for files that contain routes")
class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFilesMixin, BaseAgent):
class Troubleshooter(IterationPromptMixin, BaseAgent):
agent_type = "troubleshooter"
display_name = "Troubleshooter"
async def run(self) -> AgentResponse:
if self.current_state.unfinished_iterations:
if self.current_state.current_iteration.get("status") == IterationStatus.FIND_SOLUTION:
return await self.propose_solution()
else:
raise ValueError("There is unfinished iteration but it's not in FIND_SOLUTION state.")
else:
return await self.create_iteration()
async def propose_solution(self) -> AgentResponse:
user_feedback = self.current_state.current_iteration.get("user_feedback")
user_feedback_qa = self.current_state.current_iteration.get("user_feedback_qa")
bug_hunting_cycles = self.current_state.current_iteration.get("bug_hunting_cycles")
llm_solution = await self.find_solution(
user_feedback, user_feedback_qa=user_feedback_qa, bug_hunting_cycles=bug_hunting_cycles
)
self.next_state.current_iteration["description"] = llm_solution
self.next_state.current_iteration["status"] = IterationStatus.IMPLEMENT_SOLUTION
self.next_state.flag_iterations_as_modified()
return AgentResponse.done(self)
async def create_iteration(self) -> AgentResponse:
run_command = await self.get_run_command()
user_instructions = self.current_state.current_task.get("test_instructions")
@ -75,71 +48,47 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
self.next_state.flag_tasks_as_modified()
return AgentResponse.done(self)
else:
await self.ui.send_project_stage({"stage": ProjectStage.TEST_APP})
await self.ui.send_message("Test the app by following these steps:", source=pythagora_source)
await self.ui.send_test_instructions(user_instructions, project_state_id=str(self.current_state.id))
await self.send_message("Here are instruction on how to test the app:\n\n" + user_instructions)
# Developer sets iteration as "completed" when it generates the step breakdown, so we can't
# use "current_iteration" here
last_iteration = self.current_state.iterations[-1] if len(self.current_state.iterations) >= 3 else None
should_iterate, is_loop, should_redo, bug_report, change_description = await self.get_user_feedback(
should_iterate, is_loop, user_feedback = await self.get_user_feedback(
run_command,
user_instructions,
last_iteration is not None,
)
if should_redo:
# ask user to provide more info
task_redo_info_question = await self.ask_question(
"Please provide more information about the task you want to redo",
buttons_only=False,
)
if task_redo_info_question.text:
self.next_state.current_task["redo_human_instructions"] = task_redo_info_question.text
return AgentResponse.done(self)
if not should_iterate:
# User tested and reported no problems, we're done with the task
return await self.complete_task()
user_feedback = bug_report or change_description
user_feedback_qa = None # await self.generate_bug_report(run_command, user_instructions, user_feedback)
user_feedback_qa = await self.generate_bug_report(run_command, user_instructions, user_feedback)
if is_loop:
if last_iteration is not None and last_iteration.get("alternative_solutions"):
if last_iteration["alternative_solutions"]:
# If we already have alternative solutions, it means we were already in a loop.
return self.try_next_alternative_solution(user_feedback, user_feedback_qa)
else:
# Newly detected loop
iteration_status = IterationStatus.PROBLEM_SOLVER
# Newly detected loop, set up an empty new iteration to trigger ProblemSolver
llm_solution = ""
await self.trace_loop("loop-feedback")
elif bug_report is not None:
iteration_status = IterationStatus.HUNTING_FOR_BUG
else:
# should be - elif change_description is not None: - but to prevent bugs with the extension
# this might be caused if we show the input field instead of buttons
iteration_status = IterationStatus.NEW_FEATURE_REQUESTED
llm_solution = await self.find_solution(user_feedback, user_feedback_qa=user_feedback_qa)
self.next_state.iterations = self.current_state.iterations + [
{
"id": uuid4().hex,
"user_feedback": user_feedback,
"user_feedback_qa": user_feedback_qa,
"description": None,
"description": llm_solution,
"alternative_solutions": [],
# FIXME - this is incorrect if this is a new problem; otherwise we could
# just count the iterations
"attempts": 1,
"status": iteration_status,
"bug_hunting_cycles": [],
"completed": False,
}
]
self.next_state.flag_iterations_as_modified()
if len(self.next_state.iterations) == LOOP_THRESHOLD:
await self.trace_loop("loop-start")
@ -154,7 +103,7 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
await self.trace_loop("loop-end")
current_task_index1 = self.current_state.tasks.index(self.current_state.current_task) + 1
self.next_state.action = TS_TASK_REVIEWED.format(current_task_index1)
self.next_state.action = f"Task #{current_task_index1} reviewed"
self.next_state.set_current_task_status(TaskStatus.REVIEWED)
return AgentResponse.done(self)
@ -163,15 +112,6 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
task = self.current_state.current_task
current_task_index = self.current_state.tasks.index(task)
related_api_endpoints = task.get("related_api_endpoints", [])
# TODO: Temp fix for old projects
if not (
related_api_endpoints
and len(related_api_endpoints) > 0
and all(isinstance(api, dict) and "endpoint" in api for api in related_api_endpoints)
):
related_api_endpoints = []
return (
AgentConvo(self)
.template(
@ -179,7 +119,6 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
task=task,
iteration=None,
current_task_index=current_task_index,
related_api_endpoints=related_api_endpoints,
)
.assistant(self.current_state.current_task["instructions"])
)
@ -190,51 +129,41 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
await self.send_message("Figuring out how to run the app ...")
llm = self.get_llm(TROUBLESHOOTER_GET_RUN_COMMAND)
llm = self.get_llm()
convo = self._get_task_convo().template("get_run_command")
# Although the prompt is explicit about not using "```", LLM may still return it
llm_response: str = await llm(convo, temperature=0, parser=OptionalCodeBlockParser())
if len(llm_response) < 5:
llm_response = ""
self.next_state.run_command = llm_response
return llm_response
async def get_user_instructions(self) -> Optional[str]:
await self.send_message("### Determining how to test the app ...")
await self.send_message("Determining how to test the app ...")
route_files = await self._get_route_files()
current_task = self.current_state.current_task
llm = self.get_llm()
convo = (
self._get_task_convo()
.template(
"define_user_review_goal",
task=current_task,
route_files=route_files,
current_task_index=self.current_state.tasks.index(current_task),
)
.require_schema(TestSteps)
convo = self._get_task_convo().template(
"define_user_review_goal", task=self.current_state.current_task, route_files=route_files
)
user_instructions: TestSteps = await llm(convo, parser=JSONParser(TestSteps))
user_instructions: str = await llm(convo)
if len(user_instructions.steps) == 0:
await self.ui.send_message(
"No testing required for this task, moving on to the next one.", source=pythagora_source
)
user_instructions = user_instructions.strip()
if user_instructions.lower() == "done":
log.debug(f"Nothing to do for user testing for task {self.current_state.current_task['description']}")
return None
user_instructions = json.dumps([test.dict() for test in user_instructions.steps])
return user_instructions
async def _get_route_files(self) -> list[File]:
"""Returns the list of file paths that have routes defined in them."""
llm = self.get_llm()
convo = AgentConvo(self).template("get_route_files").require_schema(RouteFilePaths)
llm = self.get_llm(ROUTE_FILES_AGENT_NAME)
convo = (
self._get_task_convo()
.template("get_route_files", task=self.current_state.current_task)
.require_schema(RouteFilePaths)
)
file_list = await llm(convo, parser=JSONParser(RouteFilePaths))
route_files: set[str] = set(file_list.files)
@ -246,7 +175,7 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
run_command: str,
user_instructions: str,
last_iteration: Optional[dict],
) -> tuple[bool, bool, bool, str, str]:
) -> tuple[bool, bool, str, str]:
"""
Ask the user to test the app and provide feedback.
@ -258,98 +187,51 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
If "is_loop" is True, Pythagora is stuck in a loop and needs to consider alternative solutions.
If "should_redo" is True, the user wants to redo the task and we need to reset the task and start over.
The last element in the tuple is the user feedback, which may be empty if the user provided no
feedback (eg. if they just clicked on "Continue" or "Start Pair Programming").
feedback (eg. if they just clicked on "Continue" or "I'm stuck in a loop").
"""
bug_report = None
change_description = None
hint = None
test_message = "Can you check if the app works please?"
if user_instructions:
hint = " Here is a description of what should be working:\n\n" + user_instructions
is_loop = False
should_iterate = True
should_redo = False
extra_info = {"restart_app": True} if not self.current_state.iterations else None
if run_command:
await self.ui.send_run_command(run_command)
while True:
await self.ui.send_project_stage({"stage": ProjectStage.GET_USER_FEEDBACK})
buttons = {"continue": "continue"}
if last_iteration:
buttons["loop"] = "I'm stuck in a loop"
test_message = TS_APP_WORKING
if user_instructions:
hint = " Here is a description of what should be working:\n\n" + user_instructions
user_response = await self.ask_question(test_message, buttons=buttons, default="continue", hint=hint)
if user_response.button == "continue" or user_response.cancelled:
return False, False, ""
if run_command:
await self.ui.send_run_command(run_command)
buttons = {
"continue": "Everything works",
"bug": "There is an issue",
"change": "I want to make a change",
}
if not self.current_state.current_task.get("hardcoded", False):
buttons["redo"] = "Redo task"
user_response = await self.ask_question(
test_message,
buttons=buttons,
default="continue",
buttons_only=True,
hint=hint,
extra_info=extra_info,
if user_response.button == "loop":
await telemetry.trace_code_event(
"stuck-in-loop",
{
"clicked": True,
"task_index": self.current_state.tasks.index(self.current_state.current_task) + 1,
"num_tasks": len(self.current_state.tasks),
"num_epics": len(self.current_state.epics),
"num_iterations": len(self.current_state.iterations),
"num_steps": len(self.current_state.steps),
"architecture": {
"system_dependencies": self.current_state.specification.system_dependencies,
"app_dependencies": self.current_state.specification.package_dependencies,
},
},
)
extra_info = None
if user_response.button == "redo":
should_redo = True
break
if user_response.button == "continue" or user_response.cancelled:
should_iterate = False
break
return True, True, ""
elif user_response.button == "change":
await self.ui.send_project_stage({"stage": ProjectStage.DESCRIBE_CHANGE})
user_description = await self.ask_question(
"Please describe the change you want to make to the project specification (one at a time)",
buttons={"back": "Back"},
)
if user_description.button == "back":
continue
change_description = user_description.text
await self.get_relevant_files_parallel(user_feedback=change_description)
break
elif user_response.button == "bug":
await self.ui.send_project_stage({"stage": ProjectStage.DESCRIBE_ISSUE})
user_description = await self.ask_question(
TS_DESCRIBE_ISSUE,
extra_info={"collect_logs": True},
buttons={"back": "Back"},
)
if user_description.button == "back":
continue
bug_report = user_description.text
await self.ui.send_project_stage(
{
"bug_fix_attempt": 1,
}
)
await self.get_relevant_files_parallel(user_feedback=bug_report)
break
elif user_response.text and isinstance(user_response.text, str):
bug_report = user_response.text
await self.get_relevant_files_parallel(user_feedback=bug_report)
break
return should_iterate, is_loop, should_redo, bug_report, change_description
return True, False, user_response.text
def try_next_alternative_solution(self, user_feedback: str, user_feedback_qa: list[str]) -> AgentResponse:
"""
Call the ProblemSolver to try an alternative solution.
Stores the user feedback and sets iteration state so that ProblemSolver will be triggered.
Stores the user feedback and sets iteration state (not completed, no description)
so that ProblemSolver will be triggered.
:param user_feedback: User feedback to store in the iteration state.
:param user_feedback_qa: Additional questions/answers about the problem.
@ -360,9 +242,9 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
next_state_iteration["user_feedback"] = user_feedback
next_state_iteration["user_feedback_qa"] = user_feedback_qa
next_state_iteration["attempts"] += 1
next_state_iteration["status"] = IterationStatus.PROBLEM_SOLVER
next_state_iteration["completed"] = False
self.next_state.flag_iterations_as_modified()
self.next_state.action = TS_ALT_SOLUTION.format(next_state_iteration["attempts"])
self.next_state.action = f"Alternative solution (attempt #{next_state_iteration['attempts']})"
return AgentResponse.done(self)
async def generate_bug_report(
@ -380,7 +262,7 @@ class Troubleshooter(ChatWithBreakdownMixin, IterationPromptMixin, RelevantFiles
:return: Additional questions and answers to generate a better bug report.
"""
additional_qa = []
llm = self.get_llm(stream_output=True)
llm = self.get_llm()
convo = (
AgentConvo(self)
.template(

@ -1,174 +0,0 @@
import json
from urllib.parse import urljoin
from uuid import uuid4
import httpx
from sqlalchemy import inspect
from core.agents.base import BaseAgent
from core.agents.response import AgentResponse
from core.cli.helpers import capture_exception
from core.config import PYTHAGORA_API
from core.config.actions import FE_INIT
from core.db.models import KnowledgeBase
from core.log import get_logger
from core.telemetry import telemetry
log = get_logger(__name__)
class Wizard(BaseAgent):
agent_type = "wizard"
display_name = "Wizard"
async def run(self) -> AgentResponse:
success = await self.init_template()
if not success:
return AgentResponse.exit(self)
return AgentResponse.create_specification(self)
async def init_template(self) -> bool:
"""
Sets up the frontend
:return: AgentResponse.done(self)
"""
self.next_state.action = FE_INIT
self.state_manager.template = {}
options = {}
auth_data = {}
if self.state_manager.project.project_type == "swagger":
while True:
try:
docs = await self.ask_question(
"Paste the OpenAPI/Swagger JSON or YAML docs here",
allow_empty=False,
verbose=True,
)
success, options["external_api_url"], options["types"] = await self.upload_docs(docs.text)
if not success:
await self.send_message("Please try creating a new project.")
return False
else:
break
except Exception as e:
log.debug(f"An error occurred: {str(e)}")
await self.send_message("Please provide a valid input.")
continue
while True:
auth_type_question = await self.ask_question(
"Which authentication method does your backend use?",
buttons={
"none": "No authentication",
"api_key": "API Key",
"bearer": "HTTP Bearer (coming soon)",
"open_id_connect": "OpenID Connect (coming soon)",
"oauth2": "OAuth2 (coming soon)",
},
buttons_only=True,
default="api_key",
full_screen=True,
)
if auth_type_question.button == "api_key":
if auth_data.get("types") is None or "apiKey" not in auth_data["types"]:
addit_question = await self.ask_question(
"The API key authentication method is not supported by your backend. Do you want to continue?",
buttons_only=True,
buttons={"yes": "Yes", "no": "Go back"},
)
if addit_question.button != "yes":
continue
api_key = await self.ask_question(
"Enter your API key here. It will be saved in the .env file on the frontend.",
allow_empty=False,
verbose=True,
)
options["auth_type"] = "api_key"
options["api_key"] = api_key.text.strip()
break
elif auth_type_question.button == "none":
options["auth_type"] = "none"
break
else:
auth_type_question_trace = await self.ask_question(
"We are still working on getting this auth method implemented correctly. Can we contact you to get more info on how you would like it to work?",
allow_empty=False,
buttons={"yes": "Yes", "no": "No"},
default="yes",
buttons_only=True,
)
if auth_type_question_trace.button == "yes":
await telemetry.trace_code_event(
"swagger-auth-method",
{"type": auth_type_question.button},
)
await self.send_message("Thank you for submitting your request. We will be in touch.")
else:
options["auth_type"] = "login"
# Create a new knowledge base instance for the project state
knowledge_base = KnowledgeBase(pages=[], apis=[], user_options=options, utility_functions=[])
session = inspect(self.next_state).async_session
session.add(knowledge_base)
self.next_state.knowledge_base = knowledge_base
self.next_state.epics = [
{
"id": uuid4().hex,
"name": "Build frontend",
"source": "frontend",
"description": "",
"messages": [],
"summary": None,
"completed": False,
}
]
return True
async def upload_docs(self, docs: str) -> (bool, str, list):
error = None
url = urljoin(PYTHAGORA_API, "rag/upload")
for attempt in range(3):
log.debug(f"Uploading docs to RAG service... attempt {attempt}")
try:
async with httpx.AsyncClient(
transport=httpx.AsyncHTTPTransport(), timeout=httpx.Timeout(30.0, connect=5.0)
) as client:
resp = await client.post(
url,
json={
"text": docs.strip(),
"project_id": str(self.state_manager.project.id),
},
headers={"Authorization": f"Bearer {self.state_manager.get_access_token()}"},
)
if resp.status_code == 200:
log.debug("Uploading docs to RAG service successful")
resp_body = json.loads(resp.text)
return True, resp_body["external_api_url"], resp_body["types"]
elif resp.status_code == 403:
log.debug("Uploading docs to RAG service failed, trying to refresh token")
access_token = await self.ui.send_token_expired()
self.state_manager.update_access_token(access_token)
else:
try:
error = resp.json()["error"]
except Exception as e:
capture_exception(e)
error = e
log.debug(f"Uploading docs to RAG service failed: {error}")
except Exception as e:
log.warning(f"Attempt {attempt + 1} failed: {e}", exc_info=True)
capture_exception(e)
await self.ui.send_message(
f"An error occurred while uploading the docs. Error: {error if error else 'unknown'}",
)
return False

@ -3,59 +3,21 @@ import os
import os.path
import sys
from argparse import ArgumentParser, ArgumentTypeError, Namespace
from difflib import unified_diff
from typing import Optional
from urllib.parse import urlparse
from uuid import UUID
from core.config import Config, LLMProvider, LocalIPCConfig, ProviderConfig, UIAdapter, get_config, loader
from core.config.actions import (
BH_ADDITIONAL_FEEDBACK,
BH_HUMAN_TEST_AGAIN,
BH_IS_BUG_FIXED,
BH_START_BUG_HUNT,
BH_START_USER_TEST,
BH_STARTING_PAIR_PROGRAMMING,
BH_WAIT_BUG_REP_INSTRUCTIONS,
CM_UPDATE_FILES,
DEV_EXECUTE_TASK,
DEV_TASK_BREAKDOWN,
DEV_TASK_START,
DEV_TROUBLESHOOT,
FE_CHANGE_REQ,
FE_DONE_WITH_UI,
HUMAN_INTERVENTION_QUESTION,
MIX_BREAKDOWN_CHAT_PROMPT,
RUN_COMMAND,
TC_TASK_DONE,
TL_EDIT_DEV_PLAN,
TS_APP_WORKING,
TS_DESCRIBE_ISSUE,
)
from core.config.env_importer import import_from_dotenv
from core.config.version import get_version
from core.db.models import ProjectState
from core.db.models.project_state import TaskStatus
from core.db.session import SessionManager
from core.db.setup import run_migrations
from core.llm.parser import DescriptiveCodeBlockParser
from core.log import get_logger, setup
from core.log import setup
from core.state.state_manager import StateManager
from core.ui.base import AgentSource, UIBase, UISource
from core.ui.base import UIBase
from core.ui.console import PlainConsoleUI
from core.ui.ipc_client import IPCClientUI
from core.ui.virtual import VirtualUI
from core.utils.text import trim_logs
log = get_logger(__name__)
try:
import sentry_sdk
from sentry_sdk.integrations.asyncio import AsyncioIntegration
SENTRY_AVAILABLE = True
except ImportError:
SENTRY_AVAILABLE = False
def parse_llm_endpoint(value: str) -> Optional[tuple[LLMProvider, str]]:
@ -65,7 +27,7 @@ def parse_llm_endpoint(value: str) -> Optional[tuple[LLMProvider, str]]:
Option syntax is: --llm-endpoint <provider>:<url>
:param value: Argument value.
:return: Tuple with LLM provider and URL, or None if the option wasn't provided.
:return: Tuple with LLM provider and URL, or None if if the option wasn't provided.
"""
if not value:
return None
@ -85,74 +47,6 @@ def parse_llm_endpoint(value: str) -> Optional[tuple[LLMProvider, str]]:
return provider, url.geturl()
def get_line_changes(old_content: str, new_content: str) -> tuple[int, int]:
"""
Get the number of added and deleted lines between two files.
This uses Python difflib to produce a unified diff, then counts
the number of added and deleted lines.
:param old_content: old file content
:param new_content: new file content
:return: a tuple (added_lines, deleted_lines)
"""
from_lines = old_content.splitlines(keepends=True)
to_lines = new_content.splitlines(keepends=True)
diff_gen = unified_diff(from_lines, to_lines)
added_lines = 0
deleted_lines = 0
for line in diff_gen:
if line.startswith("+") and not line.startswith("+++"): # Exclude the file headers
added_lines += 1
elif line.startswith("-") and not line.startswith("---"): # Exclude the file headers
deleted_lines += 1
return added_lines, deleted_lines
def calculate_pr_changes(convo_entries):
"""
Calculate file changes between initial and final versions, similar to a Git pull request.
This function tracks the initial (first seen) and final (last seen) versions of each file,
and calculates the diff between those two versions, ignoring intermediate changes.
:param convo_entries: List of conversation entries containing file changes
:return: List of file changes with initial and final versions
"""
file_changes = {}
for entry in convo_entries:
if not entry.get("files"):
continue
for file_data in entry.get("files", []):
path = file_data.get("path")
if not path:
continue
if path not in file_changes:
file_changes[path] = {
"path": path,
"old_content": file_data.get("old_content", ""),
"new_content": file_data.get("new_content", ""),
}
else:
file_changes[path]["new_content"] = file_data.get("new_content", "")
for path, file_info in file_changes.items():
file_info["n_new_lines"], file_info["n_del_lines"] = get_line_changes(
old_content=file_info["old_content"], new_content=file_info["new_content"]
)
# Convert dict to list
return list(file_changes.values())
def parse_llm_key(value: str) -> Optional[tuple[LLMProvider, str]]:
"""
Parse --llm-key command-line option.
@ -201,9 +95,6 @@ def parse_arguments() -> Namespace:
--import-v0: Import data from a v0 (gpt-pilot) database with the given path
--email: User's email address, if provided
--extension-version: Version of the VSCode extension, if used
--use-git: Use Git for version control
--access-token: Access token
--initial-prompt: Initial prompt to automatically start a new project with 'node' stack
:return: Parsed arguments object.
"""
version = get_version()
@ -221,9 +112,6 @@ def parse_arguments() -> Namespace:
parser.add_argument("--project", help="Load a specific project", type=UUID, required=False)
parser.add_argument("--branch", help="Load a specific branch", type=UUID, required=False)
parser.add_argument("--step", help="Load a specific step in a project/branch", type=int, required=False)
parser.add_argument(
"--project-state-id", help="Load a specific project state in a project/branch", type=UUID, required=False
)
parser.add_argument("--delete", help="Delete a specific project", type=UUID, required=False)
parser.add_argument(
"--llm-endpoint",
@ -246,38 +134,6 @@ def parse_arguments() -> Namespace:
)
parser.add_argument("--email", help="User's email address", required=False)
parser.add_argument("--extension-version", help="Version of the VSCode extension", required=False)
parser.add_argument("--use-git", help="Use Git for version control", action="store_true", required=False)
parser.add_argument(
"--no-auto-confirm-breakdown",
help="Disable auto confirm when LLM requests user input",
action="store_false",
dest="auto_confirm_breakdown",
required=False,
)
parser.add_argument("--access-token", help="Access token", required=False)
parser.add_argument(
"--enable-api-server",
action="store_true",
default=True,
help="Enable IPC server for external clients",
)
parser.add_argument(
"--local-api-server-host",
type=str,
default="localhost",
help="Host for the IPC server (default: localhost)",
)
parser.add_argument(
"--local-api-server-port",
type=int,
default=8222,
help="Port for the IPC server (default: 8222)",
)
parser.add_argument(
"--initial-prompt",
help="Initial prompt to automatically start a new project with 'node' stack",
required=False,
)
return parser.parse_args()
@ -336,522 +192,44 @@ async def list_projects_json(db: SessionManager):
"""
sm = StateManager(db)
projects = await sm.list_projects()
projects_list = []
for row in projects:
project_id, project_name, created_at, folder_name = row
projects_list.append(
{
"id": project_id.hex,
"name": project_name,
"folder_name": folder_name,
"updated_at": created_at.isoformat(),
}
)
print(json.dumps(projects_list, indent=2, default=str))
def insert_new_task(tasks, new_task):
# Find the index of the first task with status "todo"
todo_index = -1
for i, task in enumerate(tasks):
if task.get("status") == TaskStatus.TODO:
todo_index = i
break
if todo_index != -1:
tasks.insert(todo_index, new_task)
else:
tasks.append(new_task)
return tasks
data = []
for project in projects:
last_updated = None
p = {
"name": project.name,
"id": project.id.hex,
"branches": [],
}
for branch in project.branches:
b = {
"name": branch.name,
"id": branch.id.hex,
"steps": [],
}
for state in branch.states:
if not last_updated or state.created_at > last_updated:
last_updated = state.created_at
s = {
"name": state.action or f"Step #{state.step_index}",
"step": state.step_index,
}
b["steps"].append(s)
if b["steps"]:
b["steps"][-1]["name"] = "Latest step"
p["branches"].append(b)
p["updated_at"] = last_updated.isoformat() if last_updated else None
data.append(p)
def find_task_by_id(tasks, task_id):
"""
Find a task by its ID from a list of tasks.
:param tasks: List of task objects
:param task_id: Task ID to search for
:return: Task object if found, None otherwise
"""
for task in tasks:
if task.get("id") == task_id:
return task
return None
def change_order_of_task(tasks, task_to_move, new_position):
# Remove the task from its current position
tasks.remove(task_to_move)
# Insert the task at the new position
tasks.insert(new_position, task_to_move)
return tasks
def find_first_todo_task(tasks):
"""
Find the first task with status 'todo' from a list of tasks.
:param tasks: List of task objects
:return: First task with status 'todo', or None if not found
"""
if not tasks:
return None
for task in tasks:
if task.get("status") == "todo":
return task
return None
def find_first_todo_task_index(tasks):
for i, task in enumerate(tasks):
if task["status"] == TaskStatus.TODO:
return i
return -1
def get_epic_task_number(state, current_task) -> (int, int):
epic_num = -1
task_num = -1
for task in state.tasks:
epic_n = task.get("sub_epic_id", 1) + 2
if epic_n != epic_num:
epic_num = epic_n
task_num = 1
if current_task["id"] == task["id"]:
return epic_num, task_num
task_num += 1
return epic_num, task_num
def get_source_for_history(msg_type: Optional[str] = "", question: Optional[str] = ""):
if question in [TL_EDIT_DEV_PLAN]:
return AgentSource("Tech Lead", "tech-lead")
if question in [FE_CHANGE_REQ, FE_DONE_WITH_UI]:
return AgentSource("Frontend", "frontend")
elif question in [
TS_DESCRIBE_ISSUE,
BH_HUMAN_TEST_AGAIN,
BH_IS_BUG_FIXED,
TS_APP_WORKING,
BH_ADDITIONAL_FEEDBACK,
] or msg_type in ["instructions", "bh_breakdown"]:
return AgentSource("Bug Hunter", "bug-hunter")
elif msg_type in ["bug_reproduction_instructions", "bug_description"]:
return AgentSource("Troubleshooter", "troubleshooter")
elif msg_type in ["frontend"]:
return AgentSource("Frontend", "frontend")
elif HUMAN_INTERVENTION_QUESTION in question:
return AgentSource("Human Input", "human-input")
elif RUN_COMMAND in question:
return AgentSource("Executor", "executor")
print(json.dumps(data, indent=2))
elif msg_type in ["task_description", "task_breakdown"]:
return AgentSource("Developer", "developer")
else:
return UISource("Pythagora", "pythagora")
"""
Prints the conversation history to the UI.
:param ui: UI instance to send messages to
:param convo: List of conversation messages to print
:param fake: If True, messages will NOT be sent to the extension.
"""
async def print_convo(ui: UIBase, convo: list, fake: Optional[bool] = True):
msgs = []
for msg in convo:
if "frontend" in msg:
frontend_data = msg["frontend"]
if isinstance(frontend_data, list):
for frontend_msg in frontend_data:
msgs.append(
await ui.send_message(
frontend_msg,
source=get_source_for_history(msg_type="frontend"),
project_state_id=msg["id"],
fake=fake,
)
)
else:
msgs.append(
await ui.send_message(
frontend_data,
source=get_source_for_history(msg_type="frontend"),
project_state_id=msg["id"],
fake=fake,
)
)
if "bh_breakdown" in msg:
msgs.append(
await ui.send_message(
msg["bh_breakdown"],
source=get_source_for_history(msg_type="bh_breakdown"),
project_state_id=msg["id"],
fake=fake,
)
)
if "task_description" in msg:
msgs.append(
await ui.send_message(
msg["task_description"],
source=get_source_for_history(msg_type="task_description"),
project_state_id=msg["id"],
fake=fake,
)
)
if "task_breakdown" in msg:
msgs.append(
await ui.send_message(
msg["task_breakdown"],
source=get_source_for_history(msg_type="task_breakdown"),
project_state_id=msg["id"],
fake=fake,
)
)
if "test_instructions" in msg:
msgs.append(
await ui.send_test_instructions(msg["test_instructions"], project_state_id=msg["id"], fake=fake)
)
if "bh_testing_instructions" in msg:
msgs.append(
await ui.send_test_instructions(msg["bh_testing_instructions"], project_state_id=msg["id"], fake=fake)
)
if "files" in msg:
for f in msg["files"]:
msgs.append(await ui.send_file_status(f["path"], "done", fake=fake))
msgs.append(
await ui.generate_diff(
file_path=f["path"],
old_content=f.get("old_content", ""),
new_content=f.get("new_content", ""),
n_new_lines=f["diff"][0],
n_del_lines=f["diff"][1],
fake=fake,
)
)
if "user_inputs" in msg and msg["user_inputs"]:
for input_item in msg["user_inputs"]:
if "question" in input_item:
msgs.append(
await ui.send_message(
input_item["question"],
source=get_source_for_history(question=input_item["question"]),
project_state_id=msg["id"],
fake=fake,
)
)
if "answer" in input_item:
if input_item["question"] != TL_EDIT_DEV_PLAN:
msgs.append(
await ui.send_user_input_history(
input_item["answer"], project_state_id=msg["id"], fake=fake
)
)
return msgs
async def load_convo(
sm: StateManager,
project_id: Optional[UUID] = None,
branch_id: Optional[UUID] = None,
project_states: Optional[list] = None,
) -> list:
"""
Loads the conversation from an existing project.
returns: list of dictionaries with the conversation history
async def list_projects(db: SessionManager):
"""
convo = []
if not branch_id:
branch_id = sm.current_state.branch_id
if project_states is not None and len(project_states) == 0:
return convo
if not project_states:
project_states = await sm.get_project_states(project_id, branch_id)
task_counter = 1
fe_printed_msgs = []
fe_commands = []
for i, state in enumerate(project_states):
convo_el = {}
convo_el["id"] = str(state.id) if state.step_index >= 3 else None
user_inputs = await sm.find_user_input(state, branch_id)
if state.tasks and state.current_task:
task_counter = state.tasks.index(state.current_task) + 1
if user_inputs:
convo_el["user_inputs"] = []
for ui in user_inputs:
if ui.question:
if ui.question == MIX_BREAKDOWN_CHAT_PROMPT:
if len(state.iterations) > 0:
# as it's not available in the current state, take the next state's description - that is the bug description!
next_state = project_states[i + 1] if i + 1 < len(project_states) else None
if next_state is not None and next_state.iterations is not None:
si = next_state.iterations[-1]
if si is not None:
if si.get("description", None) is not None:
convo_el["bh_breakdown"] = si["description"]
else:
# if there are no iterations, it means developer made task breakdown, take the next state's first task with status = todo
next_state = project_states[i + 1] if i + 1 < len(project_states) else None
if next_state is not None:
task = find_first_todo_task(next_state.tasks)
if task and task.get("test_instructions", None) is not None:
convo_el["test_instructions"] = task["test_instructions"]
if task and task.get("instructions", None) is not None:
convo_el["task_breakdown"] = task["instructions"]
# skip parsing that questions and its answers due to the fact that we do not keep states inside breakdown convo
break
if ui.question == BH_HUMAN_TEST_AGAIN:
if len(state.iterations) > 0:
si = state.iterations[-1]
if si is not None:
if si.get("bug_reproduction_description", None) is not None:
convo_el["bh_testing_instructions"] = si["bug_reproduction_description"]
if ui.question == TS_APP_WORKING:
task = find_first_todo_task(state.tasks)
if task:
if task.get("test_instructions", None) is not None:
convo_el["test_instructions"] = task["test_instructions"]
if ui.question == DEV_EXECUTE_TASK:
task = find_first_todo_task(state.tasks)
if task:
if task.get("description", None) is not None:
convo_el["task_description"] = f"Task #{task_counter} - " + task["description"]
answer = trim_logs(ui.answer_text) if ui.answer_text is not None else ui.answer_button
if answer == "bug":
answer = "There is an issue"
elif answer == "change":
answer = "I want to make a change"
convo_el["user_inputs"].append({"question": ui.question, "answer": answer})
if len(state.epics[-1].get("messages", [])) > 0:
if convo_el.get("user_inputs", None) is None:
convo_el["user_inputs"] = []
parser = DescriptiveCodeBlockParser()
for msg in state.epics[-1].get("messages", []):
if msg.get("role") == "assistant" and msg["content"] not in fe_printed_msgs:
if "frontend" not in convo_el:
convo_el["frontend"] = []
convo_el["frontend"].append(msg["content"])
fe_printed_msgs.append(msg["content"])
blocks = parser(msg.get("content", ""))
files_dict = {}
for block in blocks.blocks:
description = block.description.strip()
content = block.content.strip()
# Split description into lines and check the last line for file path
description_lines = description.split("\n")
last_line = description_lines[-1].strip()
if "file:" in last_line:
# Extract file path from the last line - get everything after "file:"
file_path = last_line[last_line.index("file:") + 5 :].strip()
file_path = file_path.strip("\"'`")
# Skip empty file paths
if file_path.strip() == "":
continue
new_content = content
old_content = sm.current_state.get_file_content_by_path(file_path)
diff = get_line_changes(
old_content=old_content,
new_content=new_content,
)
if diff != (0, 0):
files_dict[file_path] = {
"path": file_path,
"old_content": old_content,
"new_content": new_content,
"diff": diff,
}
elif "command:" in last_line:
commands = content.strip().split("\n")
for command in commands:
command = command.strip()
if command and command not in fe_commands:
if "user_inputs" not in convo_el:
convo_el["user_inputs"] = []
convo_el["user_inputs"].append(
{
"question": f"{RUN_COMMAND} {command}",
"answer": "yes",
}
)
fe_commands.append(command)
convo_el["files"] = list(files_dict.values())
if state.action is not None:
if state.action == DEV_TROUBLESHOOT.format(task_counter):
if state.iterations is not None and len(state.iterations) > 0:
si = state.iterations[-1]
if si is not None:
if si.get("user_feedback", None) is not None:
convo_el["user_feedback"] = si["user_feedback"]
if si.get("description", None) is not None:
convo_el["description"] = si["description"]
elif state.action == DEV_TASK_BREAKDOWN.format(task_counter):
if state.tasks and len(state.tasks) >= task_counter:
task = state.current_task
if task.get("description", None) is not None:
convo_el["task_description"] = f"Task #{task_counter} - " + task["description"]
if task.get("instructions", None) is not None:
convo_el["task_breakdown"] = task["instructions"]
elif state.action == TC_TASK_DONE.format(task_counter):
if state.tasks:
next_task = find_first_todo_task(state.tasks)
if next_task is not None and next_task.get("description", None) is not None:
convo_el["task_description"] = f"Task #{task_counter} - " + next_task["description"]
elif state.action == DEV_TASK_START.format(task_counter):
if state.tasks and len(state.tasks) >= task_counter:
task = state.current_task
if task.get("instructions", None) is not None:
convo_el["task_breakdown"] = task["instructions"]
elif state.action == CM_UPDATE_FILES:
files_dict = {}
for steps in state.steps:
if "save_file" in steps and "path" in steps["save_file"]:
path = steps["save_file"]["path"]
current_file = await sm.get_file_for_project(state.id, path)
prev_file = (
await sm.get_file_for_project(state.prev_state_id, path)
if state.prev_state_id is not None
else None
)
old_content = prev_file.content.content if prev_file and prev_file.content else ""
new_content = current_file.content.content if current_file and current_file.content else ""
diff = get_line_changes(
old_content=old_content,
new_content=new_content,
)
# Only add file if it has changes
if diff != (0, 0):
files_dict[path] = {
"path": path,
"old_content": old_content,
"new_content": new_content,
"diff": diff,
"bug_hunter": len(state.iterations) > 0
and len(state.iterations[-1].get("bug_hunting_cycles", [])) > 0,
}
convo_el["files"] = list(files_dict.values())
if state.iterations is not None and len(state.iterations) > 0:
si = state.iterations[-1]
if state.action == BH_START_BUG_HUNT.format(task_counter):
if si.get("user_feedback", None) is not None:
convo_el["user_feedback"] = si["user_feedback"]
if si.get("description", None) is not None:
convo_el["description"] = si["description"]
elif state.action == BH_WAIT_BUG_REP_INSTRUCTIONS.format(task_counter):
for si in state.iterations:
if si.get("bug_reproduction_description", None) is not None:
convo_el["bug_reproduction_description"] = si["bug_reproduction_description"]
elif state.action == BH_START_USER_TEST.format(task_counter):
if si.get("bug_hunting_cycles", None) is not None:
cycle = si["bug_hunting_cycles"][-1]
if cycle is not None:
if "user_feedback" in cycle and cycle["user_feedback"] is not None:
convo_el["user_feedback"] = cycle["user_feedback"]
if (
"human_readable_instructions" in cycle
and cycle["human_readable_instructions"] is not None
):
convo_el["human_readable_instructions"] = cycle["human_readable_instructions"]
elif state.action == BH_STARTING_PAIR_PROGRAMMING.format(task_counter):
if "user_feedback" in si and si["user_feedback"] is not None:
convo_el["user_feedback"] = si["user_feedback"]
if "initial_explanation" in si and si["initial_explanation"] is not None:
convo_el["initial_explanation"] = si["initial_explanation"]
convo_el["action"] = state.action
convo.append(convo_el)
return convo
def init_sentry():
if SENTRY_AVAILABLE:
sentry_sdk.init(
dsn="https://4101633bc5560bae67d6eab013ba9686@o4508731634221056.ingest.us.sentry.io/4508732401909760",
send_default_pii=True,
traces_sample_rate=1.0,
integrations=[AsyncioIntegration()],
)
def capture_exception(exc: Exception):
if SENTRY_AVAILABLE:
init_sentry()
sentry_sdk.capture_exception(exc)
async def list_projects_branches_states(db: SessionManager):
"""
List all projects in the database, including their branches and project states
List all projects in the database.
"""
sm = StateManager(db)
projects = await sm.list_projects_with_branches_states()
projects = await sm.list_projects()
print(f"Available projects ({len(projects)}):")
for project in projects:
@ -866,8 +244,7 @@ async def load_project(
project_id: Optional[UUID] = None,
branch_id: Optional[UUID] = None,
step_index: Optional[int] = None,
project_state_id: Optional[UUID] = None,
) -> Optional[ProjectState]:
) -> bool:
"""
Load a project from the database.
@ -880,26 +257,22 @@ async def load_project(
step_txt = f" step {step_index}" if step_index else ""
if branch_id:
project_state = await sm.load_project(
branch_id=branch_id, step_index=step_index, project_state_id=project_state_id
)
project_state = await sm.load_project(branch_id=branch_id, step_index=step_index)
if project_state:
return project_state
return True
else:
print(f"Branch {branch_id}{step_txt} not found; use --list to list all projects", file=sys.stderr)
return None
return False
elif project_id:
project_state = await sm.load_project(
project_id=project_id, step_index=step_index, project_state_id=project_state_id
)
project_state = await sm.load_project(project_id=project_id, step_index=step_index)
if project_state:
return project_state
return True
else:
print(f"Project {project_id}{step_txt} not found; use --list to list all projects", file=sys.stderr)
return None
return False
return None
return False
async def delete_project(db: SessionManager, project_id: UUID) -> bool:
@ -947,16 +320,9 @@ def init() -> tuple[UIBase, SessionManager, Namespace]:
ui = PlainConsoleUI()
run_migrations(config.db)
db = SessionManager(config.db, args)
db = SessionManager(config.db)
return (ui, db, args)
__all__ = [
"parse_arguments",
"load_config",
"list_projects_json",
"list_projects_branches_states",
"load_project",
"init",
]
__all__ = ["parse_arguments", "load_config", "list_projects_json", "list_projects", "load_project", "init"]

@ -1,66 +1,22 @@
import asyncio
import atexit
import gc
import signal
import sys
import traceback
from argparse import Namespace
from asyncio import run
from core.config.actions import FE_ITERATION_DONE
try:
import sentry_sdk
SENTRY_AVAILABLE = True
except ImportError:
SENTRY_AVAILABLE = False
from core.agents.orchestrator import Orchestrator
from core.cli.helpers import (
capture_exception,
delete_project,
init,
init_sentry,
list_projects_branches_states,
list_projects_json,
load_convo,
load_project,
print_convo,
show_config,
)
from core.cli.helpers import delete_project, init, list_projects, list_projects_json, load_project, show_config
from core.config import LLMProvider, get_config
from core.db.session import SessionManager
from core.db.v0importer import LegacyDatabaseImporter
from core.llm.anthropic_client import CustomAssertionError
from core.llm.base import APIError
from core.llm.base import APIError, BaseLLMClient
from core.log import get_logger
from core.state.state_manager import StateManager
from core.telemetry import telemetry
from core.ui.api_server import IPCServer
from core.ui.base import (
UIBase,
UIClosedError,
pythagora_source,
)
from core.ui.base import UIBase, UIClosedError, UserInput, pythagora_source
log = get_logger(__name__)
telemetry_sent = False
async def cleanup(ui: UIBase):
global telemetry_sent
if not telemetry_sent:
await telemetry.send()
telemetry_sent = True
await ui.stop()
def sync_cleanup(ui: UIBase):
asyncio.run(cleanup(ui))
async def run_project(sm: StateManager, ui: UIBase, args) -> bool:
async def run_project(sm: StateManager, ui: UIBase) -> bool:
"""
Work on the project.
@ -69,14 +25,13 @@ async def run_project(sm: StateManager, ui: UIBase, args) -> bool:
:param sm: State manager.
:param ui: User interface.
:param args: Command-line arguments.
:return: True if the orchestrator exited successfully, False otherwise.
"""
telemetry.set("app_id", str(sm.project.id))
telemetry.set("initial_prompt", sm.current_state.specification.description)
orca = Orchestrator(sm, ui, args=args)
orca = Orchestrator(sm, ui)
success = False
try:
success = await orca.run()
@ -86,108 +41,88 @@ async def run_project(sm: StateManager, ui: UIBase, args) -> bool:
telemetry.set("end_result", "interrupt")
await sm.rollback()
except APIError as err:
log.warning(f"an LLM API error occurred: {err.message}")
await send_error(ui, "error while calling the LLM API", err)
log.warning(f"LLM API error occurred: {err.message}")
await ui.send_message(
f"Stopping Pythagora due to an error while calling the LLM API: {err.message}",
source=pythagora_source,
)
telemetry.set("end_result", "failure:api-error")
await sm.rollback()
except CustomAssertionError as err:
log.warning(f"an Anthropic assertion error occurred: {str(err)}")
await send_error(ui, "error inside Anthropic SDK", err)
telemetry.set("end_result", "failure:assertion-error")
await sm.rollback()
except Exception as err:
log.error(f"Uncaught exception: {err}", exc_info=True)
await send_error(ui, "an error", err)
telemetry.record_crash(err)
stack_trace = telemetry.record_crash(err)
await sm.rollback()
await ui.send_message(
f"Stopping Pythagora due to error:\n\n{stack_trace}",
source=pythagora_source,
)
return success
async def send_error(ui: UIBase, error_source: str, err: Exception):
stack_trace = traceback.format_exc()
await ui.send_fatal_error(
f"Stopping Pythagora due to {error_source}:\n\n{err}",
source=pythagora_source,
extra_info={
"fatal_error": True,
"stack_trace": stack_trace,
},
)
capture_exception(err)
async def llm_api_check(ui: UIBase) -> bool:
"""
Check whether the configured LLMs are reachable.
:param ui: UI we'll use to report any issues
:return: True if all the LLMs are reachable.
"""
config = get_config()
async def start_new_project(sm: StateManager, ui: UIBase, args: Namespace = None) -> bool:
async def handler(*args, **kwargs):
pass
success = True
checked_llms: set[LLMProvider] = set()
for llm_config in config.all_llms():
if llm_config.provider in checked_llms:
continue
client_class = BaseLLMClient.for_provider(llm_config.provider)
llm_client = client_class(llm_config, stream_handler=handler, error_handler=handler)
try:
resp = await llm_client.api_check()
if not resp:
success = False
log.warning(f"API check for {llm_config.provider.value} failed.")
else:
log.info(f"API check for {llm_config.provider.value} succeeded.")
except APIError as err:
await ui.send_message(
f"API check for {llm_config.provider.value} failed with: {err}",
source=pythagora_source,
)
log.warning(f"API check for {llm_config.provider.value} failed with: {err}")
success = False
if not success:
telemetry.set("end_result", "failure:api-error")
return success
async def start_new_project(sm: StateManager, ui: UIBase) -> bool:
"""
Start a new project.
:param sm: State manager.
:param ui: User interface.
:param args: Command-line arguments.
:return: True if the project was created successfully, False otherwise.
"""
# Check if initial_prompt is provided, if so, automatically select "node"
if args and args.initial_prompt:
stack_button = "node"
await ui.send_back_logs(
[
{
"title": "",
"project_state_id": "spec",
"labels": [""],
"convo": [{"role": "assistant", "content": "Please describe the app you want to build."}],
}
]
)
else:
stack = await ui.ask_question(
"What do you want to build?",
try:
user_input = await ui.ask_question(
"What is the project name?",
allow_empty=False,
buttons={
"node": "Full stack app\n(easiest to get started)",
"swagger": "Frontend only\n(if you have backend with OpenAPI\\Swagger)",
},
buttons_only=True,
source=pythagora_source,
full_screen=True,
)
except (KeyboardInterrupt, UIClosedError):
user_input = UserInput(cancelled=True)
await ui.send_back_logs(
[
{
"title": "",
"project_state_id": "setup",
"labels": [""],
"convo": [{"role": "assistant", "content": "What do you want to build?"}],
}
]
)
if stack.button == "other":
language = await ui.ask_question(
"What language you want to use?",
allow_empty=False,
source=pythagora_source,
full_screen=True,
)
await telemetry.trace_code_event(
"stack-choice-other",
{"language": language.text},
)
await ui.send_message("Thank you for submitting your request to support other languages.")
return False
stack_button = stack.button
await telemetry.trace_code_event(
"stack-choice",
{"language": stack_button},
)
if user_input.cancelled:
return False
project_state = await sm.create_project(project_type=stack_button)
project_state = await sm.create_project(user_input.text)
return project_state is not None
@ -201,124 +136,24 @@ async def run_pythagora_session(sm: StateManager, ui: UIBase, args: Namespace):
:return: True if the application ran successfully, False otherwise.
"""
if args.project or args.branch or args.step or args.project_state_id:
if not await llm_api_check(ui):
await ui.send_message(
"Pythagora cannot start because the LLM API is not reachable.",
source=pythagora_source,
)
return False
if args.project or args.branch or args.step:
telemetry.set("is_continuation", True)
sm.fe_auto_debug = False
project_state = await load_project(sm, args.project, args.branch, args.step, args.project_state_id)
if not project_state:
success = await load_project(sm, args.project, args.branch, args.step)
if not success:
return False
# SPECIFICATION
fe_states = await sm.get_fe_states(limit=10)
be_back_logs, last_task_in_db = await sm.get_be_back_logs()
if sm.current_state.specification:
if not sm.current_state.specification.original_description:
spec = sm.current_state.specification
spec.description = project_state.epics[0]["description"]
spec.original_description = project_state.epics[0]["description"]
await sm.update_specification(spec)
await ui.send_front_logs_headers(
"",
["E1 / T1", "Writing Specification", "working" if fe_states == [] else "done"],
"Writing Specification",
"",
)
await ui.send_back_logs(
[
{
"project_state_id": "spec",
"disallow_reload": True,
"labels": ["E1 / T1", "Specs", "working" if fe_states == [] else "done"],
"title": "Writing Specification",
"convo": [
{
"role": "assistant",
"content": "What do you want to build?",
},
{
"role": "user",
"content": sm.current_state.specification.original_description,
},
],
"start_id": "",
"end_id": "",
}
]
)
if not fe_states and be_back_logs and not last_task_in_db:
await ui.send_message(
sm.current_state.specification.description,
extra_info={"route": "forwardToCenter", "screen": "spec"},
)
# FRONTEND
if fe_states:
status = "working" if fe_states[-1].action != FE_ITERATION_DONE else "done"
await ui.send_front_logs_headers(fe_states[0].id, ["E2 / T1", "Frontend", status], "Building Frontend", "")
await ui.send_back_logs(
[
{
"labels": ["E2 / T1", "Frontend", status],
"title": "Building Frontend",
"convo": [],
"project_state_id": fe_states[0].id,
"start_id": fe_states[0].id,
"end_id": fe_states[-1].id,
}
]
)
# BACKEND
if be_back_logs:
await ui.send_back_logs(be_back_logs)
if not be_back_logs and not last_task_in_db:
# if no backend logs AND no task is currently active -> we are on frontend -> print frontend convo history
convo = await load_convo(sm, project_states=fe_states)
await print_convo(ui=ui, convo=convo, fake=False)
# Clear fe_states from memory after conversation is loaded
del fe_states
gc.collect() # Force garbage collection to free memory immediately
elif last_task_in_db:
# Clear fe_states from memory as they're not needed for backend processing
del fe_states
gc.collect() # Force garbage collection to free memory immediately
# if there is a task in the db (we are at backend stage), print backend convo history and add task back logs and front logs headers
await ui.send_front_logs_headers(
last_task_in_db["start_id"],
last_task_in_db["labels"],
last_task_in_db["title"],
last_task_in_db.get("task_id", ""),
)
await ui.send_back_logs(
[
{
"project_state_id": last_task_in_db["start_id"],
"labels": last_task_in_db["labels"],
"title": last_task_in_db["title"],
"convo": [],
"start_id": last_task_in_db["start_id"],
"end_id": last_task_in_db["end_id"],
}
]
)
be_states = await sm.get_project_states_in_between(last_task_in_db["start_id"], last_task_in_db["end_id"])
convo = await load_convo(sm, project_states=be_states)
await print_convo(ui=ui, convo=convo, fake=False)
# Clear be_states from memory after conversation is loaded
del be_states
gc.collect() # Force garbage collection to free memory immediately
else:
sm.fe_auto_debug = True
success = await start_new_project(sm, ui, args)
success = await start_new_project(sm, ui)
if not success:
return False
return await run_project(sm, ui, args)
return await run_project(sm, ui)
async def async_main(
@ -334,10 +169,9 @@ async def async_main(
:param args: Command-line arguments.
:return: True if the application ran successfully, False otherwise.
"""
global telemetry_sent
if args.list:
await list_projects_branches_states(db)
await list_projects(db)
return True
elif args.list_json:
await list_projects_json(db)
@ -354,74 +188,19 @@ async def async_main(
return success
telemetry.set("user_contact", args.email)
if SENTRY_AVAILABLE and args.email:
init_sentry()
sentry_sdk.set_user({"email": args.email})
if args.extension_version:
log.debug(f"Extension version: {args.extension_version}")
telemetry.set("is_extension", True)
telemetry.set("extension_version", args.extension_version)
sm = StateManager(db, ui)
if args.access_token:
sm.update_access_token(args.access_token)
# Start API server if enabled in config
api_server = None
if hasattr(args, "enable_api_server") and args.enable_api_server:
api_host = getattr(args, "local_api_server_host", "localhost")
api_port = getattr(args, "local_api_server_port", 8222) # Different from client port
api_server = IPCServer(api_host, api_port, sm)
server_started = await api_server.start()
if not server_started:
log.warning(f"Failed to start API server on {api_host}:{api_port}")
if not args.auto_confirm_breakdown:
sm.auto_confirm_breakdown = False
ui_started = await ui.start()
if not ui_started:
if api_server:
await api_server.stop()
return False
telemetry.start()
def signal_handler(sig, frame):
try:
loop = asyncio.get_running_loop()
def close_all():
loop.stop()
sys.exit(0)
if not telemetry_sent:
cleanup_task = loop.create_task(cleanup(ui))
cleanup_task.add_done_callback(close_all)
else:
close_all()
except RuntimeError:
if not telemetry_sent:
sync_cleanup(ui)
sys.exit(0)
for sig in (signal.SIGINT, signal.SIGTERM):
signal.signal(sig, signal_handler)
# Register the cleanup function
atexit.register(sync_cleanup, ui)
try:
success = await run_pythagora_session(sm, ui, args)
except Exception as err:
log.error(f"Uncaught exception in main session: {err}", exc_info=True)
await send_error(ui, "an error", err)
raise
finally:
await cleanup(ui)
if api_server:
await api_server.stop()
success = await run_pythagora_session(sm, ui, args)
await telemetry.send()
await ui.stop()
return success

@ -5,8 +5,6 @@ from typing import Any, Literal, Optional, Union
from pydantic import BaseModel, ConfigDict, Field, field_validator
from typing_extensions import Annotated
from core.config.constants import LOGS_LINE_LIMIT
ROOT_DIR = abspath(join(dirname(__file__), "..", ".."))
DEFAULT_IGNORE_PATHS = [
".git",
@ -30,31 +28,16 @@ DEFAULT_IGNORE_PATHS = [
"*.csv",
"*.log",
"go.sum",
"migration_lock.toml",
]
IGNORE_SIZE_THRESHOLD = 50000 # 50K+ files are ignored by default
# Agents with sane setup in the default configuration
DEFAULT_AGENT_NAME = "default"
CODE_MONKEY_AGENT_NAME = "CodeMonkey"
CODE_REVIEW_AGENT_NAME = "CodeMonkey.code_review"
IMPLEMENT_CHANGES_AGENT_NAME = "CodeMonkey.implement_changes"
DESCRIBE_FILES_AGENT_NAME = "CodeMonkey.describe_files"
CHECK_LOGS_AGENT_NAME = "BugHunter.check_logs"
PARSE_TASK_AGENT_NAME = "Developer.parse_task"
TASK_BREAKDOWN_AGENT_NAME = "Developer.breakdown_current_task"
TROUBLESHOOTER_BUG_REPORT = "Troubleshooter.generate_bug_report"
TROUBLESHOOTER_GET_RUN_COMMAND = "Troubleshooter.get_run_command"
TROUBLESHOOTER_DEFINE_USER_REVIEW_GOAL = "Troubleshooter.define_user_review_goal"
TECH_LEAD_PLANNING = "TechLead.plan_epic"
TECH_LEAD_EPIC_BREAKDOWN = "TechLead.epic_breakdown"
SPEC_WRITER_AGENT_NAME = "SpecWriter"
GET_RELEVANT_FILES_AGENT_NAME = "get_relevant_files"
FRONTEND_AGENT_NAME = "Frontend"
ROUTE_FILES_AGENT_NAME = "Troubleshooter.get_route_files"
# Endpoint for the external documentation
EXTERNAL_DOCUMENTATION_API = "http://docs-pythagora-io-439719575.us-east-1.elb.amazonaws.com"
PYTHAGORA_API = "https://api.pythagora.ai"
class _StrictModel(BaseModel):
@ -73,7 +56,6 @@ class LLMProvider(str, Enum):
"""
OPENAI = "openai"
RELACE = "relace"
ANTHROPIC = "anthropic"
GROQ = "groq"
LM_STUDIO = "lm-studio"
@ -109,7 +91,7 @@ class ProviderConfig(_StrictModel):
ge=0.0,
)
read_timeout: float = Field(
default=60.0,
default=10.0,
description="Timeout (in seconds) for receiving a new chunk of data from the response stream",
ge=0.0,
)
@ -127,8 +109,8 @@ class AgentLLMConfig(_StrictModel):
AgentLLMConfig is not specified, default will be used.
"""
provider: Optional[LLMProvider] = Field(default=LLMProvider.OPENAI, description="LLM provider")
model: str = Field(description="Model to use", default="gpt-4o-2024-05-13")
provider: LLMProvider = LLMProvider.OPENAI
model: str = Field(description="Model to use", default="gpt-4-0125-preview")
temperature: float = Field(
default=0.5,
description="Temperature to use for sampling",
@ -164,7 +146,7 @@ class LLMConfig(_StrictModel):
ge=0.0,
)
read_timeout: float = Field(
default=60.0,
default=10.0,
description="Timeout (in seconds) for receiving a new chunk of data from the response stream",
ge=0.0,
)
@ -221,13 +203,9 @@ class LogConfig(_StrictModel):
description="Logging format",
)
output: Optional[str] = Field(
"data/pythagora.log",
"pythagora.log",
description="Output file for logs (if not specified, logs are printed to stderr)",
)
max_lines: int = Field(
LOGS_LINE_LIMIT,
description="Maximum number of lines to keep in the log file",
)
class DBConfig(_StrictModel):
@ -240,11 +218,10 @@ class DBConfig(_StrictModel):
"""
url: str = Field(
"sqlite+aiosqlite:///data/database/pythagora.db",
"sqlite+aiosqlite:///pythagora.db",
description="Database connection URL",
)
debug_sql: bool = Field(False, description="Log all SQL queries to the console")
save_llm_requests: bool = Field(False, description="Save LLM requests to db")
@field_validator("url")
@classmethod
@ -327,91 +304,12 @@ class Config(_StrictModel):
Pythagora Core configuration
"""
llm: dict[LLMProvider, ProviderConfig] = Field(
default={
LLMProvider.OPENAI: ProviderConfig(),
LLMProvider.ANTHROPIC: ProviderConfig(),
LLMProvider.RELACE: ProviderConfig(),
}
)
llm: dict[LLMProvider, ProviderConfig] = Field(default={LLMProvider.OPENAI: ProviderConfig()})
agent: dict[str, AgentLLMConfig] = Field(
default={
DEFAULT_AGENT_NAME: AgentLLMConfig(),
CHECK_LOGS_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.5,
),
CODE_MONKEY_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.0,
),
CODE_REVIEW_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-3-5-sonnet-20240620",
temperature=0.0,
),
IMPLEMENT_CHANGES_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.RELACE,
model="relace-code-merge",
temperature=0.0, # temperature is unused for relace
),
DESCRIBE_FILES_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="gpt-4o-mini-2024-07-18",
temperature=0.0,
),
FRONTEND_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.0,
),
GET_RELEVANT_FILES_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="gpt-4o-2024-05-13",
temperature=0.5,
),
PARSE_TASK_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-3-5-sonnet-20241022",
temperature=0.0,
),
SPEC_WRITER_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.0,
),
TASK_BREAKDOWN_AGENT_NAME: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.5,
),
TECH_LEAD_PLANNING: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-3-5-sonnet-20240620",
temperature=0.5,
),
TECH_LEAD_EPIC_BREAKDOWN: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-3-5-sonnet-20241022",
temperature=0.5,
),
TROUBLESHOOTER_BUG_REPORT: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.5,
),
TROUBLESHOOTER_GET_RUN_COMMAND: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.0,
),
TROUBLESHOOTER_DEFINE_USER_REVIEW_GOAL: AgentLLMConfig(
provider=LLMProvider.OPENAI,
model="claude-sonnet-4-20250514",
temperature=0.0,
),
DESCRIBE_FILES_AGENT_NAME: AgentLLMConfig(model="gpt-3.5-turbo", temperature=0.0),
ROUTE_FILES_AGENT_NAME: AgentLLMConfig(model="gpt-4o", temperature=0.0),
}
)
prompt: PromptConfig = PromptConfig()
@ -504,41 +402,13 @@ class ConfigLoader:
loader = ConfigLoader()
def adapt_for_bedrock(config: Config) -> Config:
"""
Adapt the configuration for use with Bedrock.
:param config: Configuration to adapt.
:return: Adapted configuration.
"""
if "anthropic" not in config.llm:
return config
if config.llm["anthropic"].base_url is None or "bedrock/anthropic" not in config.llm["anthropic"].base_url:
return config
replacement_map = {
"claude-3-5-sonnet-20241022": "us.anthropic.claude-3-5-sonnet-20241022-v2:0",
"claude-3-5-sonnet-20240620": "us.anthropic.claude-3-5-sonnet-20240620-v1:0",
"claude-3-sonnet-20240229": "us.anthropic.claude-3-sonnet-20240229-v1:0",
"claude-3-haiku-20240307": "us.anthropic.claude-3-haiku-20240307-v1:0",
"claude-3-opus-20240229": "us.anthropic.claude-3-opus-20240229-v1:0",
}
for agent in config.agent:
if config.agent[agent].model in replacement_map:
config.agent[agent].model = replacement_map[config.agent[agent].model]
return config
def get_config() -> Config:
"""
Return current configuration.
:return: Current configuration object.
"""
return adapt_for_bedrock(loader.config)
return loader.config
__all__ = ["loader", "get_config"]

@ -1,58 +0,0 @@
BH_START_BUG_HUNT = "Start bug hunt for task #{}"
BH_WAIT_BUG_REP_INSTRUCTIONS = "Awaiting bug reproduction instructions for task #{}"
BH_START_USER_TEST = "Start user testing for task #{}"
BH_STARTING_PAIR_PROGRAMMING = "Start pair programming for task #{}"
CM_UPDATE_FILES = "Updating files"
DEV_WAIT_TEST = "Awaiting user test"
DEV_TASK_START = "Task #{} start"
DEV_TASK_BREAKDOWN = "Task #{} breakdown"
DEV_TROUBLESHOOT = "Troubleshooting #{}"
DEV_TASK_REVIEW_FEEDBACK = "Task review feedback"
TC_TASK_DONE = "Task #{} complete"
FE_INIT = "Frontend init"
FE_START = "Frontend start"
FE_CONTINUE = "Frontend continue"
FE_ITERATION = "Frontend iteration"
FE_ITERATION_DONE = "Frontend iteration done"
TL_CREATE_INITIAL_EPIC = "Create initial project epic"
TL_CREATE_PLAN = "Create a development plan for epic: {}"
TL_START_FEATURE = "Start of feature #{}"
TL_INITIAL_PROJECT_NAME = "Initial Project"
TW_WRITE = "Write documentation"
EX_SKIP_COMMAND = 'Skip "{}"'
EX_RUN_COMMAND = 'Run "{}"'
SPEC_CREATE_STEP_NAME = "Create specification"
SPEC_CHANGE_STEP_NAME = "Change specification"
SPEC_CHANGE_FEATURE_STEP_NAME = "Change specification due to new feature"
TS_TASK_REVIEWED = "Task #{} reviewed"
TS_ALT_SOLUTION = "Alternative solution (attempt #{})"
TS_APP_WORKING = "Please check if the app is working"
PS_EPIC_COMPLETE = "Epic {} completed"
# other constants
TL_EDIT_DEV_PLAN = "Open and edit your development plan in the Progress tab"
MIX_BREAKDOWN_CHAT_PROMPT = "Are you happy with the breakdown? Now is a good time to ask questions or suggest changes."
FE_CHANGE_REQ = (
"Do you want to change anything or report a bug? Keep in mind that currently ONLY frontend is implemented."
)
FE_DONE_WITH_UI = "Are you sure you're done building the UI and want to start building the backend functionality now?"
TS_DESCRIBE_ISSUE = "Please describe the issue you found (one at a time) and share any relevant server logs"
BH_HUMAN_TEST_AGAIN = "Please test the app again."
BH_IS_BUG_FIXED = "Is the bug you reported fixed now?"
BH_ADDITIONAL_FEEDBACK = "Please add any additional feedback that could help Pythagora solve this bug"
HUMAN_INTERVENTION_QUESTION = "I need human intervention"
CONTINUE_WHEN_DONE = 'When you\'re done, just click "Continue"'
RUN_COMMAND = "Can I run command:"
DEV_EXECUTE_TASK = "Do you want to execute the above task?"

@ -1,2 +0,0 @@
CONVO_ITERATIONS_LIMIT = 8
LOGS_LINE_LIMIT = 20000

@ -1,37 +0,0 @@
PROBLEM_IDENTIFIED = "PROBLEM_IDENTIFIED"
ADD_LOGS = "ADD_LOGS"
ALWAYS_RELEVANT_FILES = [
"client/src/App.tsx",
]
GITIGNORE_CONTENT = """# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
# SQLite databases, data files
*.db
*.csv
# Keep environment variables out of version control
.env
"""

@ -39,30 +39,6 @@ def get_git_commit() -> Optional[str]:
return f.read().strip()
def get_git_branch() -> Optional[str]:
"""
Return the current git branch name (if running from a repo).
:return: branch name or None if not on a branch or not a git repo
"""
if not isdir(GIT_DIR_PATH):
return None
git_head = join(GIT_DIR_PATH, "HEAD")
if not isfile(git_head):
return None
with open(git_head, "r", encoding="utf-8") as f:
ref = f.read().strip()
if ref.startswith("ref: "):
# Example: ref: refs/heads/main
ref_path = ref[5:]
if ref_path.startswith("refs/heads/"):
return ref_path[len("refs/heads/") :]
return None
def get_package_version() -> str:
"""
Get package version as defined pyproject.toml.
@ -107,4 +83,4 @@ def get_version() -> str:
return version
__all__ = ["get_version", "get_git_branch"]
__all__ = ["get_version"]

@ -61,7 +61,7 @@ version_path_separator = os
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = sqlite:///data/database/pythagora.db
sqlalchemy.url = sqlite:///pythagora.db
[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run

@ -1,90 +0,0 @@
#!/usr/bin/env python
import os
import sqlite3
from pathlib import Path
from typing import Optional
from alembic import command
from alembic.config import Config
from alembic.script import ScriptDirectory
def get_latest_revision(alembic_cfg: Config) -> Optional[str]:
"""Get the most recent revision from available migration files."""
script = ScriptDirectory.from_config(alembic_cfg)
if script.get_heads():
return script.get_heads()[0]
return None
def fix_alembic_version(db_path: str, version: str) -> None:
"""Manually update alembic_version table to the specified version."""
conn = sqlite3.connect(db_path)
try:
cursor = conn.cursor()
cursor.execute("UPDATE alembic_version SET version_num = ?", (version,))
conn.commit()
print(f"Successfully updated alembic_version to {version}")
except sqlite3.Error as e:
print(f"Error updating database: {e}")
raise
finally:
conn.close()
def main():
# Get the project root directory (where core/ is located)
project_root = Path(__file__).parent.parent.parent
# Configure alembic
alembic_ini = os.path.join(project_root, "core", "db", "alembic.ini")
if not os.path.exists(alembic_ini):
print(f"Error: Could not find alembic.ini at {alembic_ini}")
return 1
alembic_cfg = Config(alembic_ini)
# Get the database path from alembic.ini
db_url = alembic_cfg.get_main_option("sqlalchemy.url")
if not db_url.startswith("sqlite:///"):
print("Error: This script only works with SQLite databases")
return 1
db_path = db_url.replace("sqlite:///", "")
db_path = os.path.join(project_root, db_path)
if not os.path.exists(db_path):
print(f"Database file not found at {db_path}")
create_new = input("Would you like to create a new database? (y/n): ")
if create_new.lower() == "y":
print("Creating new database and running migrations...")
command.upgrade(alembic_cfg, "head")
print("Done!")
return 0
return 1
# Get the latest available revision
latest_revision = get_latest_revision(alembic_cfg)
if not latest_revision:
print("Error: No migration versions found")
return 1
print(f"Latest available revision: {latest_revision}")
try:
# Update the version in the database
fix_alembic_version(db_path, latest_revision)
# Run migrations to ensure database schema is up to date
print("Running migrations to ensure database schema is current...")
command.upgrade(alembic_cfg, "head")
print("Database successfully fixed and upgraded!")
return 0
except Exception as e:
print(f"Error: {e}")
return 1
if __name__ == "__main__":
exit(main())

@ -1,54 +0,0 @@
"""move metadata from file to file content table
Revision ID: 0173e14719aa
Revises: 3968d770dced
Create Date: 2025-05-15 15:33:03.084670
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "0173e14719aa"
down_revision: Union[str, None] = "3968d770dced"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Add meta column to file_contents
with op.batch_alter_table("file_contents", schema=None) as batch_op:
batch_op.add_column(sa.Column("meta", sa.JSON(), server_default="{}", nullable=False))
# Copy data from files.meta to file_contents.meta
op.execute("""
UPDATE file_contents
SET meta = files.meta
FROM files
WHERE file_contents.id = files.content_id
""")
# Drop meta column from files
with op.batch_alter_table("files", schema=None) as batch_op:
batch_op.drop_column("meta")
def downgrade() -> None:
# Add meta column back to files
with op.batch_alter_table("files", schema=None) as batch_op:
batch_op.add_column(sa.Column("meta", sa.JSON(), server_default="{}", nullable=False))
# Copy data from file_contents.meta back to files.meta
op.execute("""
UPDATE files
SET meta = file_contents.meta
FROM file_contents
WHERE files.content_id = file_contents.id
""")
# Drop meta column from file_contents
with op.batch_alter_table("file_contents", schema=None) as batch_op:
batch_op.drop_column("meta")

@ -1,34 +0,0 @@
"""vacuum database
Revision ID: 0173e14719ab
Revises: 69e50fdaf067
Create Date: 2025-05-15 15:33:03.084670
"""
from typing import Sequence, Union
from alembic import op
from sqlalchemy import text
# revision identifiers, used by Alembic.
revision: str = "0173e14719ab"
down_revision: Union[str, None] = "69e50fdaf067"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Get connection
connection = op.get_bind()
# Temporarily disable journal mode
connection.execute(text("PRAGMA journal_mode = OFF"))
# Run VACUUM
connection.execute(text("VACUUM"))
def downgrade() -> None:
# VACUUM is not reversible
pass

@ -1,36 +0,0 @@
"""refactor specification.template to specification.templates
Revision ID: 08d71952ec2f
Revises: ff891d366761
Create Date: 2024-06-14 18:23:09.070736
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "08d71952ec2f"
down_revision: Union[str, None] = "ff891d366761"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("specifications", schema=None) as batch_op:
batch_op.add_column(sa.Column("templates", sa.JSON(), nullable=True))
batch_op.drop_column("template")
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("specifications", schema=None) as batch_op:
batch_op.add_column(sa.Column("template", sa.VARCHAR(), nullable=True))
batch_op.drop_column("templates")
# ### end Alembic commands ###

@ -1,34 +0,0 @@
"""Add project type to project
Revision ID: 3968d770dced
Revises: f708791b9270
Create Date: 2025-02-15 10:30:13.163098
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "3968d770dced"
down_revision: Union[str, None] = "f708791b9270"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("projects", schema=None) as batch_op:
batch_op.add_column(sa.Column("project_type", sa.String(), nullable=False, server_default="node"))
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("projects", schema=None) as batch_op:
batch_op.drop_column("project_type")
# ### end Alembic commands ###

@ -1,48 +0,0 @@
"""Add chat messages and convos
Revision ID: 675268601278
Revises: 0173e14719ab
Create Date: 2025-05-14 10:38:19.130649
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy import func
# revision identifiers, used by Alembic.
revision: str = "675268601278"
down_revision: Union[str, None] = "0173e14719ab"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Create chat_convos table
op.create_table(
"chat_convos",
sa.Column("id", sa.Integer(), autoincrement=True, primary_key=True),
sa.Column("convo_id", sa.Uuid(), nullable=False, unique=True),
sa.Column(
"project_state_id", sa.Uuid(), sa.ForeignKey("project_states.id", ondelete="CASCADE"), nullable=False
),
sa.Column("created_at", sa.DateTime, server_default=func.now(), nullable=False),
)
# Create chat_messages table
op.create_table(
"chat_messages",
sa.Column("id", sa.Uuid(), primary_key=True, nullable=False),
sa.Column("convo_id", sa.Uuid(), sa.ForeignKey("chat_convos.convo_id", ondelete="CASCADE"), nullable=False),
sa.Column("created_at", sa.DateTime, server_default=func.now(), nullable=False),
sa.Column("message_type", sa.String(), nullable=False),
sa.Column("message", sa.String(), nullable=False),
sa.Column("prev_message_id", sa.Uuid(), sa.ForeignKey("chat_messages.id", ondelete="SET NULL"), nullable=True),
)
def downgrade() -> None:
op.drop_table("chat_messages")
op.drop_table("chat_convos")

@ -1,143 +0,0 @@
"""move knowledge base to separate table
Revision ID: 69e50fdaf067
Revises: 0173e14719aa
Create Date: 2025-05-15 17:27:50.312917
"""
import json
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
from sqlalchemy import JSON, Column, Integer, MetaData, Table, insert, text
from sqlalchemy.dialects import sqlite
# revision identifiers, used by Alembic.
revision: str = "69e50fdaf067"
down_revision: Union[str, None] = "0173e14719aa"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Create the new knowledge_bases table
op.create_table(
"knowledge_bases",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("pages", sa.JSON(), server_default="[]", nullable=False),
sa.Column("apis", sa.JSON(), server_default="[]", nullable=False),
sa.Column("user_options", sa.JSON(), server_default="{}", nullable=False),
sa.Column("utility_functions", sa.JSON(), server_default="[]", nullable=False),
sa.PrimaryKeyConstraint("id", name=op.f("pk_knowledge_bases")),
)
# Add knowledge_base_id column to project_states
with op.batch_alter_table("project_states", schema=None) as batch_op:
batch_op.add_column(sa.Column("knowledge_base_id", sa.Integer(), nullable=True))
# Get connection for data migration
connection = op.get_bind()
# Create a table object for knowledge_bases
metadata = MetaData()
knowledge_bases = Table(
"knowledge_bases",
metadata,
Column("id", Integer, primary_key=True),
Column("pages", JSON),
Column("apis", JSON),
Column("user_options", JSON),
Column("utility_functions", JSON),
)
# Keep track of unique knowledge bases to avoid redundancy
kb_cache = {}
# Migrate data from old knowledge_base column to new table
project_states = connection.execute(text("SELECT id, knowledge_base FROM project_states")).fetchall()
for state_id, kb_data_str in project_states:
if kb_data_str:
try:
# Parse the JSON string into a dictionary
kb_data = json.loads(kb_data_str) if isinstance(kb_data_str, str) else kb_data_str
except (json.JSONDecodeError, TypeError):
# If parsing fails, use empty defaults
kb_data = {}
# Create a cache key from the knowledge base content
cache_key = json.dumps(
{
"pages": kb_data.get("pages", []),
"apis": kb_data.get("apis", []),
"user_options": kb_data.get("user_options", {}),
"utility_functions": kb_data.get("utility_functions", []),
},
sort_keys=True,
)
if cache_key not in kb_cache:
# Insert new knowledge base record
stmt = insert(knowledge_bases).values(
pages=kb_data.get("pages", []),
apis=kb_data.get("apis", []),
user_options=kb_data.get("user_options", {}),
utility_functions=kb_data.get("utility_functions", []),
)
result = connection.execute(stmt)
kb_cache[cache_key] = result.inserted_primary_key[0]
# Update project state to reference the knowledge base
kb_id = kb_cache[cache_key]
connection.execute(
text("UPDATE project_states SET knowledge_base_id = :kb_id WHERE id = :state_id"),
{"kb_id": kb_id, "state_id": state_id},
)
# Make knowledge_base_id not nullable and add foreign key constraint
with op.batch_alter_table("project_states", schema=None) as batch_op:
batch_op.alter_column("knowledge_base_id", nullable=False)
batch_op.create_foreign_key(
batch_op.f("fk_project_states_knowledge_base_id_knowledge_bases"),
"knowledge_bases",
["knowledge_base_id"],
["id"],
)
batch_op.drop_column("knowledge_base")
# Clean up llm_requests table
op.execute("DELETE FROM llm_requests")
def downgrade() -> None:
# Add back the knowledge_base column
with op.batch_alter_table("project_states", schema=None) as batch_op:
batch_op.add_column(sa.Column("knowledge_base", sqlite.JSON(), server_default=sa.text("'{}'"), nullable=False))
# Get connection for data migration
connection = op.get_bind()
# Migrate data back from knowledge_bases table to project_states
results = connection.execute(
text("""
SELECT ps.id, kb.pages, kb.apis, kb.user_options, kb.utility_functions
FROM project_states ps
JOIN knowledge_bases kb ON ps.knowledge_base_id = kb.id
""")
).fetchall()
for state_id, pages, apis, user_options, utility_functions in results:
# Create the knowledge base data structure
kb_data = {"pages": pages, "apis": apis, "user_options": user_options, "utility_functions": utility_functions}
connection.execute(
text("UPDATE project_states SET knowledge_base = :kb_data WHERE id = :state_id"),
{"kb_data": json.dumps(kb_data), "state_id": state_id},
)
# Remove the knowledge_base_id column and knowledge_bases table
with op.batch_alter_table("project_states", schema=None) as batch_op:
batch_op.drop_constraint(batch_op.f("fk_project_states_knowledge_base_id_knowledge_bases"), type_="foreignkey")
batch_op.drop_column("knowledge_base_id")
op.drop_table("knowledge_bases")

@ -1,36 +0,0 @@
"""Add original description and template summary fields to specifications
Revision ID: c8905d4ce784
Revises: 08d71952ec2f
Create Date: 2024-07-25 19:24:23.808237
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "c8905d4ce784"
down_revision: Union[str, None] = "08d71952ec2f"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("specifications", schema=None) as batch_op:
batch_op.add_column(sa.Column("original_description", sa.String(), nullable=True))
batch_op.add_column(sa.Column("template_summary", sa.String(), nullable=True))
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("specifications", schema=None) as batch_op:
batch_op.drop_column("template_summary")
batch_op.drop_column("original_description")
# ### end Alembic commands ###

@ -1,34 +0,0 @@
"""Adding knowledge_base field to ProjectState
Revision ID: f708791b9270
Revises: c8905d4ce784
Create Date: 2024-12-22 12:13:14.979169
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "f708791b9270"
down_revision: Union[str, None] = "c8905d4ce784"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("project_states", schema=None) as batch_op:
batch_op.add_column(sa.Column("knowledge_base", sa.JSON(), server_default="{}", nullable=False))
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("project_states", schema=None) as batch_op:
batch_op.drop_column("knowledge_base")
# ### end Alembic commands ###

@ -5,12 +5,9 @@
from .base import Base
from .branch import Branch
from .chat_convo import ChatConvo
from .chat_message import ChatMessage
from .exec_log import ExecLog
from .file import File
from .file_content import FileContent
from .knowledge_base import KnowledgeBase
from .llm_request import LLMRequest
from .project import Project
from .project_state import ProjectState
@ -24,12 +21,9 @@ __all__ = [
"ExecLog",
"File",
"FileContent",
"KnowledgeBase",
"LLMRequest",
"Project",
"ProjectState",
"Specification",
"UserInput",
"ChatConvo",
"ChatMessage",
]

@ -1,48 +0,0 @@
from datetime import datetime
from typing import TYPE_CHECKING, Optional
from uuid import UUID, uuid4
from sqlalchemy import ForeignKey, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.sql import func
from core.db.models import Base
if TYPE_CHECKING:
from core.db.models import ChatMessage, ProjectState
class ChatConvo(Base):
__tablename__ = "chat_convos"
id: Mapped[int] = mapped_column(primary_key=True)
convo_id: Mapped[UUID] = mapped_column(default=uuid4, unique=True)
project_state_id: Mapped[UUID] = mapped_column(ForeignKey("project_states.id", ondelete="CASCADE"))
created_at: Mapped[datetime] = mapped_column(server_default=func.now())
# Relationships
project_state: Mapped["ProjectState"] = relationship(back_populates="chat_convos", lazy="selectin")
messages: Mapped[list["ChatMessage"]] = relationship(
back_populates="convo", cascade="all,delete-orphan", lazy="selectin"
)
@staticmethod
async def get_chat_history(session: AsyncSession, convo_id) -> list["ChatMessage"]:
from core.db.models import ChatMessage
result = await session.execute(select(ChatMessage).where(ChatMessage.convo_id == convo_id))
return result.scalars().all()
@staticmethod
async def get_project_state_for_convo_id(session: AsyncSession, convo_id) -> Optional["ProjectState"]:
from core.db.models import ChatConvo, ProjectState
result = await session.execute(select(ChatConvo).where(ChatConvo.convo_id == convo_id))
chat_convo = result.scalars().first()
if not chat_convo:
return None
result = await session.execute(select(ProjectState).where(ProjectState.id == chat_convo.project_state_id))
return result.scalars().one_or_none()

@ -1,23 +0,0 @@
from datetime import datetime
from typing import Optional
from uuid import UUID, uuid4
from sqlalchemy import ForeignKey
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.sql import func
from core.db.models import Base, ChatConvo
class ChatMessage(Base):
__tablename__ = "chat_messages"
id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4)
convo_id: Mapped[UUID] = mapped_column(ForeignKey("chat_convos.convo_id", ondelete="CASCADE"))
created_at: Mapped[datetime] = mapped_column(server_default=func.now())
message_type: Mapped[str] = mapped_column()
message: Mapped[str] = mapped_column()
prev_message_id: Mapped[Optional[UUID]] = mapped_column(ForeignKey("chat_messages.id", ondelete="SET NULL"))
# Relationships
convo: Mapped["ChatConvo"] = relationship(back_populates="messages", lazy="selectin")

@ -1,9 +1,7 @@
import re
from typing import TYPE_CHECKING, Optional
from uuid import UUID
from sqlalchemy import ForeignKey, UniqueConstraint, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import ForeignKey, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column, relationship
from core.db.models import Base
@ -23,6 +21,7 @@ class File(Base):
# Attributes
path: Mapped[str] = mapped_column()
meta: Mapped[dict] = mapped_column(default=dict, server_default="{}")
# Relationships
project_state: Mapped[Optional["ProjectState"]] = relationship(back_populates="files", lazy="raise")
@ -40,46 +39,5 @@ class File(Base):
project_state=None,
content_id=self.content_id,
path=self.path,
meta=self.meta,
)
@staticmethod
async def get_referencing_files(session: "AsyncSession", project_state, file_path_to_search) -> list["File"]:
results = await session.execute(select(File).where(File.project_state_id == project_state.id))
all_files = results.scalars().all()
file_to_search = None
for file in all_files:
if file.path == file_path_to_search:
file_to_search = file
all_files.remove(file)
break
if file_to_search is None:
return []
referencing_files = []
target_file_name = file_path_to_search.split("/")[-1].split(".")[0]
import_regex = re.compile(
rf"import.*from\s+['\"](\.?/?(?:{re.escape(target_file_name)}|{re.escape('/api' + '/' + target_file_name)}))(?:['\"])[;]*"
)
# Extract function names from the target file
function_names = set()
for match in re.finditer(r"export\s+const\s+(\w+)\s*=", file_to_search.content.content):
function_names.add(match.group(1))
function_names_list = list(function_names)
direct_function_call_regex = None
if function_names_list:
direct_function_call_regex = re.compile(rf"({'|'.join(function_names_list)})\(")
for file in all_files:
if import_regex.search(file.content.content):
referencing_files.append(file)
elif any(fn in file.content.content for fn in function_names_list):
referencing_files.append(file)
elif direct_function_call_regex and direct_function_call_regex.search(file.content.content):
referencing_files.append(file)
return referencing_files

@ -18,13 +18,12 @@ class FileContent(Base):
# Attributes
content: Mapped[str] = mapped_column()
meta: Mapped[dict] = mapped_column(default=dict, server_default="{}")
# Relationships
files: Mapped[list["File"]] = relationship(back_populates="content", lazy="raise")
@classmethod
async def store(cls, session: AsyncSession, hash: str, content: str, meta: dict = None) -> "FileContent":
async def store(cls, session: AsyncSession, hash: str, content: str) -> "FileContent":
"""
Store the file content in the database.
@ -35,17 +34,14 @@ class FileContent(Base):
:param session: The database session.
:param hash: The hash of the file content, used as an unique ID.
:param content: The file content as unicode string.
:param meta: Optional metadata for the file content.
:return: The file content object.
"""
result = await session.execute(select(FileContent).where(FileContent.id == hash))
fc = result.scalar_one_or_none()
if fc is not None:
if meta is not None:
fc.meta = meta
return fc
fc = cls(id=hash, content=content, meta=meta or {})
fc = cls(id=hash, content=content)
session.add(fc)
return fc

@ -1,61 +0,0 @@
from copy import deepcopy
from typing import TYPE_CHECKING
from sqlalchemy import JSON, delete, distinct, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Mapped, mapped_column, relationship
from core.db.models.base import Base
if TYPE_CHECKING:
from core.db.models import ProjectState
class KnowledgeBase(Base):
"""Model for storing project knowledge base.
This model stores various pieces of project-related information:
- pages: List of implemented frontend pages
- apis: List of API endpoints and their implementation status
- user_options: User configuration options for the project
- utility_functions: List of utility functions with their status, input/output values
"""
__tablename__ = "knowledge_bases"
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
pages: Mapped[list[str]] = mapped_column(JSON, default=list, server_default="[]")
apis: Mapped[list[dict]] = mapped_column(JSON, default=list, server_default="[]")
user_options: Mapped[dict] = mapped_column(JSON, default=dict, server_default="{}")
utility_functions: Mapped[list[dict]] = mapped_column(JSON, default=list, server_default="[]")
# Relationships
project_states: Mapped[list["ProjectState"]] = relationship(back_populates="knowledge_base", lazy="raise")
def clone(self) -> "KnowledgeBase":
"""
Clone the knowledge base.
Creates a new KnowledgeBase instance with the same data but new ID.
Used when the knowledge base needs to be modified to maintain immutability
of previous states.
"""
return KnowledgeBase(
pages=deepcopy(self.pages),
apis=deepcopy(self.apis),
user_options=deepcopy(self.user_options),
utility_functions=deepcopy(self.utility_functions),
)
@classmethod
async def delete_orphans(cls, session: AsyncSession):
"""
Delete KnowledgeBase objects that are not referenced by any ProjectState object.
:param session: The database session.
"""
from core.db.models import ProjectState
await session.execute(
delete(KnowledgeBase).where(~KnowledgeBase.id.in_(select(distinct(ProjectState.knowledge_base_id))))
)

@ -3,7 +3,6 @@ from typing import TYPE_CHECKING, Optional
from uuid import UUID
from sqlalchemy import ForeignKey, inspect
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.sql import func
@ -60,7 +59,7 @@ class LLMRequest(Base):
:param request_log: Request log.
:return: Newly created LLM request log in the database.
"""
session: AsyncSession = inspect(project_state).async_session
session = inspect(project_state).async_session
obj = cls(
project_state=project_state,

@ -4,12 +4,12 @@ from typing import TYPE_CHECKING, Optional, Union
from unicodedata import normalize
from uuid import UUID, uuid4
from sqlalchemy import Row, and_, delete, inspect, select
from sqlalchemy import and_, delete, inspect, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Mapped, mapped_column, relationship, selectinload
from sqlalchemy.sql import func
from core.db.models import Base, File
from core.db.models import Base
if TYPE_CHECKING:
from core.db.models import Branch
@ -27,7 +27,6 @@ class Project(Base):
folder_name: Mapped[str] = mapped_column(
default=lambda context: Project.get_folder_from_project_name(context.get_current_parameters()["name"])
)
project_type: Mapped[str] = mapped_column(default="node")
# Relationships
branches: Mapped[list["Branch"]] = relationship(back_populates="project", cascade="all", lazy="raise")
@ -47,31 +46,6 @@ class Project(Base):
result = await session.execute(select(Project).where(Project.id == project_id))
return result.scalar_one_or_none()
@staticmethod
async def rename(session: "AsyncSession", id: UUID, name: str, dir_name: str) -> Optional["Project"]:
"""
Rename a project and update its folder name.
:param session: The SQLAlchemy session.
:param id: The project ID.
:param name: The new project name.
:param dir_name: The new folder name for the project.
:return: The updated Project object if found, None otherwise.
"""
# Get the project by ID
query = select(Project).where(Project.id == id)
result = await session.execute(query)
project = result.scalar_one_or_none()
if project is None:
return None
# Update project name and dir name
project.name = name
project.folder_name = dir_name
return project
async def get_branch(self, name: Optional[str] = None) -> Optional["Branch"]:
"""
Get a project branch by name.
@ -93,28 +67,7 @@ class Project(Base):
return result.scalar_one_or_none()
@staticmethod
async def get_file_for_project(session: AsyncSession, project_state_id: UUID, path: str) -> Optional["File"]:
file_result = await session.execute(
select(File).where(File.project_state_id == project_state_id, File.path == path)
)
return file_result.scalar_one_or_none()
@staticmethod
async def get_branches_for_project_id(session: AsyncSession, project_id: UUID) -> list["Branch"]:
from core.db.models import Branch
branch_result = await session.execute(select(Branch).where(Branch.project_id == project_id))
return branch_result.scalars().all()
@staticmethod
async def get_all_projects(session: "AsyncSession") -> list[Row]:
query = select(Project.id, Project.name, Project.created_at, Project.folder_name).order_by(Project.name)
result = await session.execute(query)
return result.fetchall()
@staticmethod
async def get_all_projects_with_branches_states(session: "AsyncSession") -> list["Project"]:
async def get_all_projects(session: "AsyncSession") -> list["Project"]:
"""
Get all projects.

@ -1,30 +1,19 @@
from copy import deepcopy
from datetime import datetime
from typing import TYPE_CHECKING, Optional, Union
from typing import TYPE_CHECKING, Optional
from uuid import UUID, uuid4
from sqlalchemy import ForeignKey, UniqueConstraint, and_, delete, inspect, or_, select
from sqlalchemy import ForeignKey, UniqueConstraint, delete, inspect
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import Mapped, load_only, mapped_column, relationship
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.orm.attributes import flag_modified
from sqlalchemy.sql import func
from core.config.actions import FE_START, PS_EPIC_COMPLETE
from core.db.models import Base, FileContent
from core.db.models import Base
from core.log import get_logger
if TYPE_CHECKING:
from core.db.models import (
Branch,
ChatConvo,
ExecLog,
File,
FileContent,
KnowledgeBase,
LLMRequest,
Specification,
UserInput,
)
from core.db.models import Branch, ExecLog, File, FileContent, LLMRequest, Specification, UserInput
log = get_logger(__name__)
@ -36,26 +25,11 @@ class TaskStatus:
IN_PROGRESS = "in_progress"
REVIEWED = "reviewed"
DOCUMENTED = "documented"
EPIC_UPDATED = "epic_updated"
DONE = "done"
SKIPPED = "skipped"
class IterationStatus:
"""Status of an iteration."""
HUNTING_FOR_BUG = "check_logs"
AWAITING_LOGGING = "awaiting_logging"
AWAITING_USER_TEST = "awaiting_user_test"
AWAITING_BUG_FIX = "awaiting_bug_fix"
AWAITING_BUG_REPRODUCTION = "awaiting_bug_reproduction"
IMPLEMENT_SOLUTION = "implement_solution"
FIND_SOLUTION = "find_solution"
PROBLEM_SOLVER = "problem_solver"
NEW_FEATURE_REQUESTED = "new_feature_requested"
START_PAIR_PROGRAMMING = "start_pair_programming"
DONE = "done"
class ProjectState(Base):
__tablename__ = "project_states"
__table_args__ = (
@ -69,7 +43,6 @@ class ProjectState(Base):
branch_id: Mapped[UUID] = mapped_column(ForeignKey("branches.id", ondelete="CASCADE"))
prev_state_id: Mapped[Optional[UUID]] = mapped_column(ForeignKey("project_states.id", ondelete="CASCADE"))
specification_id: Mapped[int] = mapped_column(ForeignKey("specifications.id"))
knowledge_base_id: Mapped[int] = mapped_column(ForeignKey("knowledge_bases.id"))
# Attributes
created_at: Mapped[datetime] = mapped_column(server_default=func.now())
@ -100,13 +73,9 @@ class ProjectState(Base):
cascade="all,delete-orphan",
)
specification: Mapped["Specification"] = relationship(back_populates="project_states", lazy="selectin")
knowledge_base: Mapped["KnowledgeBase"] = relationship(back_populates="project_states", lazy="selectin")
llm_requests: Mapped[list["LLMRequest"]] = relationship(back_populates="project_state", cascade="all", lazy="raise")
user_inputs: Mapped[list["UserInput"]] = relationship(back_populates="project_state", cascade="all", lazy="raise")
exec_logs: Mapped[list["ExecLog"]] = relationship(back_populates="project_state", cascade="all", lazy="raise")
chat_convos: Mapped[list["ChatConvo"]] = relationship(
back_populates="project_state", cascade="all,delete-orphan", lazy="raise"
)
@property
def unfinished_steps(self) -> list[dict]:
@ -136,11 +105,7 @@ class ProjectState(Base):
:return: List of unfinished iterations.
"""
if not self.iterations:
return []
return [
iteration for iteration in self.iterations if iteration.get("status") not in (None, IterationStatus.DONE)
]
return [iteration for iteration in self.iterations if not iteration.get("completed")]
@property
def current_iteration(self) -> Optional[dict]:
@ -161,8 +126,6 @@ class ProjectState(Base):
:return: List of unfinished tasks.
"""
if not self.tasks:
return []
return [task for task in self.tasks if task.get("status") != TaskStatus.DONE]
@property
@ -224,48 +187,14 @@ class ProjectState(Base):
:param branch: The branch to create the state for.
:return: The new ProjectState object.
"""
from core.db.models import KnowledgeBase, Specification
from core.db.models import Specification
return ProjectState(
branch=branch,
specification=Specification(),
knowledge_base=KnowledgeBase(),
step_index=1,
action="Initial project state",
)
@staticmethod
async def get_project_states(
session: "AsyncSession",
project_id: Optional[UUID] = None,
branch_id: Optional[UUID] = None,
) -> list["ProjectState"]:
from core.db.models import Branch, ProjectState
branch = None
limit = 100
if branch_id:
branch = await session.execute(select(Branch).where(Branch.id == branch_id))
branch = branch.scalar_one_or_none()
elif project_id:
branch = await session.execute(select(Branch).where(Branch.project_id == project_id))
branch = branch.scalar_one_or_none()
if branch:
query = (
select(ProjectState)
.where(ProjectState.branch_id == branch.id)
.order_by(ProjectState.step_index.desc()) # Get the latest 100 states
.limit(limit)
)
project_states_result = await session.execute(query)
project_states = project_states_result.scalars().all()
return sorted(project_states, key=lambda x: x.step_index)
return []
async def create_next_state(self) -> "ProjectState":
"""
Create the next project state for the branch.
@ -291,7 +220,6 @@ class ProjectState(Base):
tasks=deepcopy(self.tasks),
steps=deepcopy(self.steps),
iterations=deepcopy(self.iterations),
knowledge_base=self.knowledge_base,
files=[],
relevant_files=deepcopy(self.relevant_files),
modified_files=deepcopy(self.modified_files),
@ -308,20 +236,17 @@ class ProjectState(Base):
for file in await self.awaitable_attrs.files:
clone = file.clone()
new_state.files.append(clone)
# Load content for the clone using the same content_id
result = await session.execute(select(FileContent).where(FileContent.id == file.content_id))
clone.content = result.scalar_one_or_none()
return new_state
def complete_step(self, step_type: str):
def complete_step(self):
if not self.unfinished_steps:
raise ValueError("There are no unfinished steps to complete")
if "next_state" in self.__dict__:
raise ValueError("Current state is read-only (already has a next state).")
log.debug(f"Completing step {self.unfinished_steps[0]['type']}")
self.get_steps_of_type(step_type)[0]["completed"] = True
self.unfinished_steps[0]["completed"] = True
flag_modified(self, "steps")
def complete_task(self):
@ -352,8 +277,6 @@ class ProjectState(Base):
self.unfinished_epics[0]["completed"] = True
self.tasks = []
flag_modified(self, "epics")
if len(self.unfinished_epics) > 0:
self.next_state.action = PS_EPIC_COMPLETE.format(self.unfinished_epics[0]["name"])
def complete_iteration(self):
if not self.unfinished_iterations:
@ -362,9 +285,7 @@ class ProjectState(Base):
raise ValueError("Current state is read-only (already has a next state).")
log.debug(f"Completing iteration {self.unfinished_iterations[0]}")
self.unfinished_iterations[0]["status"] = IterationStatus.DONE
self.relevant_files = None
self.modified_files = {}
self.unfinished_iterations[0]["completed"] = True
self.flag_iterations_as_modified()
def flag_iterations_as_modified(self):
@ -387,30 +308,6 @@ class ProjectState(Base):
"""
flag_modified(self, "tasks")
def flag_epics_as_modified(self):
"""
Flag the epic field as having been modified
Used by Agents that perform modifications within the mutable epics field,
to tell the database that it was modified and should get saved (as SQLalchemy
can't detect changes in mutable fields by itself).
"""
flag_modified(self, "epics")
def flag_knowledge_base_as_modified(self):
"""
Flag the knowledge base fields as having been modified
Used by Agents that perform modifications within the mutable knowledge base fields,
to tell the database that it was modified and should get saved (as SQLalchemy
can't detect changes in mutable fields by itself).
This creates a new knowledge base instance to maintain immutability of previous states,
similar to how specification modifications are handled.
"""
# Create a new knowledge base instance with the current data
self.knowledge_base = self.knowledge_base.clone()
def set_current_task_status(self, status: str):
"""
Set the status of the current task.
@ -438,17 +335,6 @@ class ProjectState(Base):
return None
def get_file_content_by_path(self, path: str) -> Union[FileContent, str]:
"""
Get a file from the current project state, by the file path.
:param path: The file path.
:return: The file object, or None if not found.
"""
file = self.get_file_by_path(path)
return file.content.content if file else ""
def save_file(self, path: str, content: "FileContent", external: bool = False) -> "File":
"""
Save a file to the project state.
@ -491,47 +377,18 @@ class ProjectState(Base):
async def delete_after(self):
"""
Delete all states in the branch after this one, along with related data.
This includes:
- ProjectState records after this one
- Related UserInput records (including those for the current state)
- Related File records
- Orphaned FileContent records
- Orphaned KnowledgeBase records
- Orphaned Specification records
Delete all states in the branch after this one.
"""
from core.db.models import FileContent, KnowledgeBase, Specification, UserInput
session: AsyncSession = inspect(self).async_session
log.debug(f"Deleting all project states in branch {self.branch_id} after {self.id}")
# Get all project states to be deleted
states_to_delete = await session.execute(
select(ProjectState).where(
await session.execute(
delete(ProjectState).where(
ProjectState.branch_id == self.branch_id,
ProjectState.step_index > self.step_index,
)
)
states_to_delete = states_to_delete.scalars().all()
state_ids = [state.id for state in states_to_delete]
# Delete user inputs for the current state
await session.execute(delete(UserInput).where(UserInput.project_state_id == self.id))
if state_ids:
# Delete related user inputs for states to be deleted
await session.execute(delete(UserInput).where(UserInput.project_state_id.in_(state_ids)))
# Delete project states
await session.execute(delete(ProjectState).where(ProjectState.id.in_(state_ids)))
# Clean up orphaned records
await FileContent.delete_orphans(session)
await UserInput.delete_orphans(session)
await KnowledgeBase.delete_orphans(session)
await Specification.delete_orphans(session)
def get_last_iteration_steps(self) -> list:
"""
@ -558,521 +415,3 @@ class ProjectState(Base):
return len([step for step in steps if step.get("type") == "review_task"])
return 1
def get_steps_of_type(self, step_type: str) -> [dict]:
"""
Get list of unfinished steps with specific type.
:return: List of steps, or empty list if there are no unfinished steps of that type.
"""
li = self.unfinished_steps
return [step for step in li if step.get("type") == step_type] if li else []
def has_frontend(self) -> bool:
"""
Check if there is a frontend epic in the project state.
:return: True if there is a frontend epic, False otherwise.
"""
return self.epics and any(epic.get("source") == "frontend" for epic in self.epics)
# function that checks whether old project or new project is currently in frontend stage
def working_on_frontend(self) -> bool:
return self.has_frontend() and len(self.epics) == 1
def is_feature(self) -> bool:
"""
Check if the current epic is a feature.
:return: True if the current epic is a feature, False otherwise.
"""
return self.epics and self.current_epic and self.current_epic.get("source") == "feature"
@staticmethod
async def get_state_for_redo_task(session: AsyncSession, project_state: "ProjectState") -> Optional["ProjectState"]:
states_result = await session.execute(
select(ProjectState).where(
and_(
ProjectState.step_index <= project_state.step_index,
ProjectState.branch_id == project_state.branch_id,
)
)
)
result = states_result.scalars().all()
result = sorted(result, key=lambda x: x.step_index, reverse=True)
for state in result:
if state.tasks:
for task in state.tasks:
if task.get("id") == project_state.current_task.get("id") and task.get("instructions") is None:
if task.get("status") == TaskStatus.TODO:
return state
return None
@staticmethod
async def get_by_id(session: "AsyncSession", state_id: UUID) -> Optional["ProjectState"]:
"""
Retrieve a project state by its ID.
:param session: The SQLAlchemy async session.
:param state_id: The UUID of the project state to retrieve.
:return: The ProjectState object if found, None otherwise.
"""
if not state_id:
return None
query = select(ProjectState).where(ProjectState.id == state_id)
result = await session.execute(query)
return result.scalar_one_or_none()
@staticmethod
async def get_all_epics_and_tasks(session: "AsyncSession", branch_id: UUID) -> list:
epics_and_tasks = []
try:
query = (
select(ProjectState)
.options(load_only(ProjectState.id, ProjectState.epics, ProjectState.tasks))
.where(and_(ProjectState.branch_id == branch_id, ProjectState.action.isnot(None)))
)
result = await session.execute(query)
project_states = result.scalars().all()
def has_epic(epic_type: str):
return any(epic1.get("source", "") == epic_type for epic1 in epics_and_tasks)
def find_epic_by_id(epic_id: str, sub_epic_id: str):
return next(
(
epic
for epic in epics_and_tasks
if epic.get("id", "") == epic_id and epic.get("sub_epic_id", "") == sub_epic_id
),
None,
)
def find_task_in_epic(task_id: str, epic):
if not epic:
return None
return next((task for task in epic.get("tasks", []) if task.get("id", "") == task_id), None)
for state in project_states:
epics, tasks = state.epics, state.tasks
epic = epics[-1]
if epics[-1] in ["spec_writer", "frontend"]:
for epic in state.epics:
if epic["source"] == "spec_writer" and not has_epic("spec_writer"):
epics_and_tasks.insert(0, {"source": "spec_writer", "tasks": []})
if epic["source"] == "frontend" and not has_epic("frontend"):
epics_and_tasks.insert(1, {"source": "frontend", "tasks": []})
else:
for sub_epic in epic.get("sub_epics", []):
if not find_epic_by_id(epic["id"], sub_epic["id"]):
epics_and_tasks.append(
{
"id": epic["id"],
"sub_epic_id": sub_epic["id"],
"source": epic["source"],
"description": sub_epic.get("description", ""),
"tasks": [],
}
)
for task in tasks:
epic_in_list = find_epic_by_id(epic["id"], task.get("sub_epic_id"))
if not epic_in_list:
continue
task_in_epic_list = find_task_in_epic(task["id"], epic_in_list)
if not task_in_epic_list:
epic_in_list["tasks"].append(
{
"id": task.get("id"),
"status": task.get("status"),
"description": task.get("description"),
}
)
else:
# Update the status of the task if it already exists
task_in_epic_list["status"] = task.get("status")
except Exception as e:
log.error(f"Error while getting epics and tasks: {e}")
return []
return epics_and_tasks
@staticmethod
async def get_project_states_in_between(
session: "AsyncSession", branch_id: UUID, start_id: UUID, end_id: UUID, limit: Optional[int] = 100
):
query = select(ProjectState).where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.id == start_id,
)
)
result = await session.execute(query)
start_state = result.scalars().one_or_none()
query = select(ProjectState).where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.id == end_id,
)
)
result = await session.execute(query)
end_state = result.scalars().one_or_none()
if not start_state or not end_state:
log.error(f"Could not find states with IDs {start_id} and {end_id} in branch {branch_id}")
return []
query = (
select(ProjectState)
.where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.step_index >= start_state.step_index,
ProjectState.step_index <= end_state.step_index,
)
)
.order_by(ProjectState.step_index.desc())
)
if limit:
query = query.limit(limit)
result = await session.execute(query)
states = result.scalars().all()
# Since we always order by step_index desc, we need to reverse to get chronological order
return list(reversed(states))
@staticmethod
async def get_task_conversation_project_states(
session: "AsyncSession",
branch_id: UUID,
task_id: UUID,
first_last_only: bool = False,
limit: Optional[int] = 25,
) -> Optional[list["ProjectState"]]:
"""
Retrieve the conversation for the task in the project state.
:param session: The SQLAlchemy async session.
:param branch_id: The UUID of the branch.
:param task_id: The UUID of the task.
:param first_last_only: If True, return only first and last states.
:param limit: Maximum number of states to return (default 25).
:return: List of conversation messages if found, None otherwise.
"""
log.debug(
f"Getting task conversation project states for task {task_id} in branch {branch_id} with first_last_only {first_last_only} and limit {limit}"
)
# First, we need to find the start and end step indices
# Use a more efficient query that only loads necessary fields
query = (
select(ProjectState)
.options(load_only(ProjectState.id, ProjectState.step_index, ProjectState.tasks, ProjectState.action))
.where(
and_(
ProjectState.branch_id == branch_id,
or_(ProjectState.action.like("%Task #%"), ProjectState.action.like("%Create a development plan%")),
)
)
.order_by(ProjectState.step_index)
)
result = await session.execute(query)
states = result.scalars().all()
log.debug(f"Found {len(states)} states with custom action")
start_step_index = None
end_step_index = None
# for the FIRST task, it is todo in the same state as Create a development plan, while other tasks are "Task #N start" (action)
# this is done solely to be able to reload to the first task, due to the fact that we need the same project_state_id for the send_back_logs
# for the first task, we need to start from the FIRST state that has that task in TODO status
# for all other tasks, we need to start from LAST state that has that task in TODO status
for state in states:
for task in state.tasks:
if UUID(task["id"]) == task_id and task.get("status", "") == TaskStatus.TODO:
if UUID(task["id"]) == UUID(state.tasks[0]["id"]):
# First task: set start only once (first occurrence)
if start_step_index is None:
start_step_index = state.step_index
else:
# Other tasks: update start every time (last occurrence)
start_step_index = state.step_index
if UUID(task["id"]) == task_id and task.get("status", "") in [
TaskStatus.SKIPPED,
TaskStatus.DOCUMENTED,
TaskStatus.REVIEWED,
TaskStatus.DONE,
]:
end_step_index = state.step_index
if start_step_index is None:
return []
# Now build the optimized query based on what we need
if first_last_only:
# For first_last_only, we only need the first and last states
# Get first state
first_query = (
select(ProjectState)
.where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.step_index >= start_step_index,
ProjectState.step_index < end_step_index if end_step_index else True,
)
)
.order_by(ProjectState.step_index.asc())
.limit(1)
)
# Get last state (excluding the uncommitted one)
last_query = (
select(ProjectState)
.where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.step_index >= start_step_index,
ProjectState.step_index < end_step_index if end_step_index else True,
)
)
.order_by(ProjectState.step_index.desc())
.limit(2)
) # Get last 2 to exclude uncommitted
first_result = await session.execute(first_query)
last_result = await session.execute(last_query)
first_state = first_result.scalars().first()
last_states = last_result.scalars().all()
# Remove the last state (uncommitted) and get the actual last
if len(last_states) > 1:
last_state = last_states[1] # Second to last is the actual last committed
else:
last_state = first_state # Only one state
if first_state and last_state and first_state.id != last_state.id:
return [first_state, last_state]
elif first_state:
return [first_state]
else:
return []
else:
# For regular queries, apply limit at the database level
query = (
select(ProjectState)
.where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.step_index >= start_step_index,
ProjectState.step_index < end_step_index if end_step_index else True,
)
)
.order_by(ProjectState.step_index.asc())
)
if limit:
# Apply limit + 1 to account for removing the last uncommitted state
query = query.limit(limit + 1)
result = await session.execute(query)
results = result.scalars().all()
log.debug(f"Found {len(results)} states with custom action")
# Remove the last state from the list because that state is not yet committed in the database!
if results:
results = results[:-1]
return results
@staticmethod
async def get_fe_states(
session: "AsyncSession", branch_id: UUID, limit: Optional[int] = None
) -> Optional["ProjectState"]:
query = select(ProjectState).where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.action == FE_START,
)
)
result = await session.execute(query)
fe_start = result.scalars().one_or_none()
if not fe_start:
return []
query = (
select(ProjectState)
.where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.step_index >= fe_start.step_index,
ProjectState.action.like("%Frontend%"),
)
)
.order_by(ProjectState.step_index.desc())
.limit(1)
)
result = await session.execute(query)
fe_end = result.scalars().one_or_none()
query = (
select(ProjectState)
.where(
and_(
ProjectState.branch_id == branch_id,
ProjectState.step_index >= fe_start.step_index,
ProjectState.step_index <= fe_end.step_index,
)
)
.order_by(ProjectState.step_index.desc())
)
if limit:
query = query.limit(limit)
results = await session.execute(query)
states = results.scalars().all()
# Since we ordered by step_index desc and limited, we need to reverse to get chronological order
return list(reversed(states))
@staticmethod
def get_epic_task_number(state, current_task) -> (int, int):
epic_num = -1
task_num = -1
for task in state.tasks:
epic_n = task.get("sub_epic_id", 1) + 2
if epic_n != epic_num:
epic_num = epic_n
task_num = 1
if current_task["id"] == task["id"]:
return epic_num, task_num
task_num += 1
return epic_num, task_num
@staticmethod
async def get_be_back_logs(session: "AsyncSession", branch_id: UUID) -> (list[dict], dict, list["ProjectState"]):
"""
For each FINISHED task in the branch, find all project states where the task status changes. Additionally, the last task that will be returned is the one that is currently being worked on.
Returns data formatted for the UI + the project states for history convo.
:param session: The SQLAlchemy async session.
:param branch_id: The UUID of the branch.
:return: List of dicts with UI-friendly task conversation format.
"""
query = select(ProjectState).where(
and_(
ProjectState.branch_id == branch_id,
or_(ProjectState.action.like("%Task #%"), ProjectState.action.like("%Create a development plan%")),
)
)
result = await session.execute(query)
states = result.scalars().all()
log.debug(f"Found {len(states)} states in branch")
if not states:
query = select(ProjectState).where(ProjectState.branch_id == branch_id)
result = await session.execute(query)
states = result.scalars().all()
task_histories = []
def find_task_history(task_id):
for th in task_histories:
if th["task_id"] == task_id:
return th
return None
for state in states:
for task in state.tasks or []:
task_id = task.get("id")
if not task_id:
continue
th = find_task_history(task_id)
if not th:
th = {
"task_id": task_id,
"title": task.get("description"),
"labels": [],
"status": task["status"],
"start_id": state.id,
"project_state_id": state.id,
"end_id": state.id,
}
task_histories.append(th)
if task.get("status") == TaskStatus.TODO:
th["status"] = TaskStatus.TODO
th["start_id"] = state.id
th["project_state_id"] = state.id
th["end_id"] = state.id
elif task.get("status") != th["status"]:
th["status"] = task.get("status")
th["end_id"] = state.id
epic_index, task_index = ProjectState.get_epic_task_number(state, task)
th["labels"] = [
f"E{str(epic_index)} / T{task_index}",
"Backend",
"Working"
if task.get("status") in [TaskStatus.TODO, TaskStatus.IN_PROGRESS]
else "Skipped"
if task.get("status") == TaskStatus.SKIPPED
else "Done",
]
last_task = {}
# todo/in_progress can override done
# done can override todo/in_progress
# todo/in_progress can not override todo/in_progress
for th in task_histories:
if not last_task:
last_task = th
# if we have multiple tasks being Worked on (todo state) in a row, then we take the first one
# if we see a Done task, we take that one
if not (
last_task["status"] in [TaskStatus.TODO, TaskStatus.IN_PROGRESS]
and th["status"] in [TaskStatus.TODO, TaskStatus.IN_PROGRESS]
):
last_task = th
if task_histories and last_task:
task_histories = task_histories[: task_histories.index(last_task) + 1]
if last_task:
project_states = await ProjectState.get_task_conversation_project_states(
session, branch_id, UUID(last_task["task_id"])
)
if project_states:
last_task["start_id"] = project_states[0].id
last_task["project_state_id"] = project_states[0].id
last_task["end_id"] = project_states[-1].id
return task_histories, last_task

@ -1,4 +1,3 @@
from copy import deepcopy
from typing import TYPE_CHECKING, Optional
from sqlalchemy import delete, distinct, select
@ -26,14 +25,11 @@ class Specification(Base):
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True)
# Attributes
original_description: Mapped[Optional[str]] = mapped_column()
description: Mapped[str] = mapped_column(default="")
template_summary: Mapped[Optional[str]] = mapped_column()
architecture: Mapped[str] = mapped_column(default="")
system_dependencies: Mapped[list[dict]] = mapped_column(default=list)
package_dependencies: Mapped[list[dict]] = mapped_column(default=list)
templates: Mapped[Optional[dict]] = mapped_column()
template: Mapped[Optional[str]] = mapped_column()
complexity: Mapped[str] = mapped_column(server_default=Complexity.HARD)
example_project: Mapped[Optional[str]] = mapped_column()
@ -45,13 +41,11 @@ class Specification(Base):
Clone the specification.
"""
clone = Specification(
original_description=self.original_description,
description=self.description,
template_summary=self.template_summary,
architecture=self.architecture,
system_dependencies=self.system_dependencies,
package_dependencies=self.package_dependencies,
templates=deepcopy(self.templates) if self.templates else None,
template=self.template,
complexity=self.complexity,
example_project=self.example_project,
)
@ -69,20 +63,3 @@ class Specification(Base):
await session.execute(
delete(Specification).where(~Specification.id.in_(select(distinct(ProjectState.specification_id))))
)
@staticmethod
async def update_specification(session: AsyncSession, specification: "Specification") -> Optional["Specification"]:
"""
Update the specification in the database.
:param session: The database session.
:param specification: The Specification object to update.
:return: The updated Specification object or None if not found.
"""
existing_spec = await session.get(Specification, specification.id)
if existing_spec:
for key, value in specification.__dict__.items():
setattr(existing_spec, key, value)
await session.commit()
return existing_spec
return None

@ -2,8 +2,7 @@ from datetime import datetime
from typing import TYPE_CHECKING, Optional
from uuid import UUID
from sqlalchemy import ForeignKey, and_, delete, inspect, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import ForeignKey, inspect
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.sql import func
@ -58,24 +57,3 @@ class UserInput(Base):
)
session.add(obj)
return obj
@staticmethod
async def find_user_inputs(session: AsyncSession, project_state, branch_id) -> Optional[list["UserInput"]]:
from core.db.models import UserInput
user_input = await session.execute(
select(UserInput).where(
and_(UserInput.branch_id == branch_id, UserInput.project_state_id == project_state.id)
)
)
user_input = user_input.scalars().all()
return user_input if len(user_input) > 0 else []
@classmethod
async def delete_orphans(cls, session: AsyncSession):
"""
Delete UserInput objects that have no associated ProjectState.
:param session: The database session.
"""
await session.execute(delete(UserInput).where(UserInput.project_state_id.is_(None)))

@ -18,7 +18,7 @@ class SessionManager:
... # Do something with the session
"""
def __init__(self, config: DBConfig, args=None):
def __init__(self, config: DBConfig):
"""
Initialize the session manager with the given configuration.
@ -31,8 +31,7 @@ class SessionManager:
self.SessionClass = async_sessionmaker(self.engine, expire_on_commit=False)
self.session = None
self.recursion_depth = 0
email_is_pythagora = args is not None and args.email is not None and args.email.endswith("@pythagora.ai")
self.save_llm_requests = config.save_llm_requests or email_is_pythagora
event.listen(self.engine.sync_engine, "connect", self._on_connect)
def _on_connect(self, dbapi_connection, _):

@ -179,7 +179,7 @@ class LegacyDatabaseImporter:
log.info(f"Importing app {app_info['name']} (id={app_id}) ...")
async with self.session_manager as session:
project = Project(id=UUID(app_id), name=app_info["name"], project_type="node")
project = Project(id=UUID(app_id), name=app_info["name"])
branch = Branch(project=project)
state = ProjectState.create_initial_state(branch)

@ -1,7 +1,6 @@
import asyncio
import datetime
import zoneinfo
from typing import Optional, Tuple
from typing import Optional
from anthropic import AsyncAnthropic, RateLimitError
from httpx import Timeout
@ -16,11 +15,6 @@ log = get_logger(__name__)
# Maximum number of tokens supported by Anthropic Claude 3
MAX_TOKENS = 4096
MAX_TOKENS_SONNET = 8192
class CustomAssertionError(Exception):
pass
class AnthropicClient(BaseLLMClient):
@ -43,7 +37,7 @@ class AnthropicClient(BaseLLMClient):
Adapt the conversation messages to the format expected by the Anthropic Claude model.
Claude only recognizes "user" and "assistant" roles, and requires them to be switched
for each message (i.e. no consecutive messages from the same role).
for each message (ie. no consecutive messages from the same role).
:param convo: Conversation to adapt.
:return: Adapted conversation messages.
@ -66,60 +60,39 @@ class AnthropicClient(BaseLLMClient):
return messages
async def _make_request(
self, convo: Convo, temperature: Optional[float] = None, json_mode: bool = False, retry_count: int = 1
) -> Tuple[str, int, int]:
async def single_attempt() -> Tuple[str, int, int]:
messages = self._adapt_messages(convo)
completion_kwargs = {
"max_tokens": MAX_TOKENS,
"model": self.config.model,
"messages": messages,
"temperature": self.config.temperature if temperature is None else temperature,
}
if "trybricks" in self.config.base_url:
completion_kwargs["extra_headers"] = {"x-request-timeout": f"{int(float(self.config.read_timeout))}s"}
if "bedrock/anthropic" in self.config.base_url:
completion_kwargs["extra_headers"] = {"anthropic-version": "bedrock-2023-05-31"}
if "sonnet" in self.config.model:
completion_kwargs["max_tokens"] = MAX_TOKENS_SONNET
if json_mode:
completion_kwargs["response_format"] = {"type": "json_object"}
response = []
async with self.client.messages.stream(**completion_kwargs) as stream:
async for content in stream.text_stream:
response.append(content)
if self.stream_handler:
await self.stream_handler(content)
try:
final_message = await stream.get_final_message()
final_message.content # Access content to verify it exists
except AssertionError:
log.debug("Anthropic package AssertionError")
raise CustomAssertionError("No final message received.")
response_str = "".join(response)
# Tell the stream handler we're done
if self.stream_handler:
await self.stream_handler(None)
return response_str, final_message.usage.input_tokens, final_message.usage.output_tokens
for attempt in range(retry_count + 1):
try:
return await single_attempt()
except CustomAssertionError as e:
if attempt == retry_count: # If this was our last attempt
raise CustomAssertionError(f"Request failed after {retry_count + 1} attempts: {str(e)}")
# Add a small delay before retrying
await asyncio.sleep(1)
continue
self,
convo: Convo,
temperature: Optional[float] = None,
json_mode: bool = False,
) -> tuple[str, int, int]:
messages = self._adapt_messages(convo)
completion_kwargs = {
"max_tokens": MAX_TOKENS,
"model": self.config.model,
"messages": messages,
"temperature": self.config.temperature if temperature is None else temperature,
}
if json_mode:
completion_kwargs["response_format"] = {"type": "json_object"}
response = []
async with self.client.messages.stream(**completion_kwargs) as stream:
async for content in stream.text_stream:
response.append(content)
if self.stream_handler:
await self.stream_handler(content)
# TODO: get tokens from the final message
final_message = await stream.get_final_message()
final_message.content
response_str = "".join(response)
# Tell the stream handler we're done
if self.stream_handler:
await self.stream_handler(None)
return response_str, final_message.usage.input_tokens, final_message.usage.output_tokens
def rate_limit_sleep(self, err: RateLimitError) -> Optional[datetime.timedelta]:
"""
@ -143,11 +116,7 @@ class AnthropicClient(BaseLLMClient):
except ValueError:
return datetime.timedelta(seconds=5)
try:
now = datetime.datetime.now(tz=zoneinfo.ZoneInfo("UTC"))
except zoneinfo.ZoneInfoNotFoundError:
now = datetime.datetime.now(tz=datetime.timezone.utc)
now = datetime.datetime.now(tz=zoneinfo.ZoneInfo("UTC"))
return reset_time - now

@ -1,25 +1,18 @@
import asyncio
import datetime
import json
import sys
from enum import Enum
from time import time
from typing import Any, Callable, Optional, Tuple
import httpx
import tiktoken
from httpx import AsyncClient
from core.config import LLMConfig, LLMProvider
from core.llm.convo import Convo
from core.llm.request_log import LLMRequestLog, LLMRequestStatus
from core.log import get_logger
from core.state.state_manager import StateManager
from core.ui.base import UIBase, pythagora_source
from core.utils.text import trim_logs
log = get_logger(__name__)
tokenizer = tiktoken.get_encoding("cl100k_base")
class LLMError(str, Enum):
@ -55,11 +48,9 @@ class BaseLLMClient:
def __init__(
self,
config: LLMConfig,
state_manager: StateManager,
*,
stream_handler: Optional[Callable] = None,
error_handler: Optional[Callable] = None,
ui: Optional[UIBase] = None,
):
"""
Initialize the client with the given configuration.
@ -70,8 +61,6 @@ class BaseLLMClient:
self.config = config
self.stream_handler = stream_handler
self.error_handler = error_handler
self.ui = ui
self.state_manager = state_manager
self._init_client()
def _init_client(self):
@ -164,27 +153,10 @@ class BaseLLMClient:
prompts=convo.prompt_log,
)
prompt_tokens = sum(3 + len(tokenizer.encode(str(msg.get("content", "")))) for msg in convo.messages)
index = -1
if prompt_tokens > 150_000:
for i, msg in enumerate(reversed(convo.messages)):
if "Here are the backend logs" in msg["content"] or "Here are the frontend logs" in msg["content"]:
index = len(convo.messages) - 1 - i
break
if index != -1:
for i, msg in enumerate(convo.messages):
if i < index:
convo.messages[i]["content"] = trim_logs(convo.messages[i]["content"])
else:
break
prompt_length_kb = len(json.dumps(convo.messages).encode("utf-8")) / 1024
log.debug(
f"Calling {self.provider.value} model {self.config.model} (temp={temperature}), prompt length: {prompt_length_kb:.1f} KB, prompt tokens (approx.): {prompt_tokens:.1f}"
f"Calling {self.provider.value} model {self.config.model} (temp={temperature}), prompt length: {prompt_length_kb:.1f} KB"
)
t0 = time()
remaining_retries = max_retries
@ -214,40 +186,6 @@ class BaseLLMClient:
response = None
try:
access_token = self.state_manager.get_access_token()
if access_token:
# Store the original client
original_client = self.client
# Copy client based on its type
if isinstance(original_client, openai.AsyncOpenAI):
self.client = openai.AsyncOpenAI(
api_key=original_client.api_key,
base_url=original_client.base_url,
timeout=original_client.timeout,
default_headers={
"Authorization": f"Bearer {access_token}",
"Timeout": str(max(self.config.connect_timeout, self.config.read_timeout)),
},
)
elif isinstance(original_client, anthropic.AsyncAnthropic):
# Create new Anthropic client with custom headers
self.client = anthropic.AsyncAnthropic(
api_key=original_client.api_key,
base_url=original_client.base_url,
timeout=original_client.timeout,
default_headers={
"Authorization": f"Bearer {access_token}",
"Timeout": str(max(self.config.connect_timeout, self.config.read_timeout)),
},
)
elif isinstance(original_client, AsyncClient):
self.client = AsyncClient()
else:
# Handle other client types or raise exception
raise ValueError(f"Unsupported client type: {type(original_client)}")
response, prompt_tokens, completion_tokens = await self._make_request(
convo,
temperature=temperature,
@ -305,57 +243,7 @@ class BaseLLMClient:
# us confirm that we've breached the token limit, but Anthropic doesn't,
# so we can't be certain that's the problem in Anthropic case.
# Here we try to detect that and tell the user what happened.
log.info(f"API status error: {err}")
if getattr(err, "status_code", None) in (401, 403):
if self.ui:
try:
await self.ui.send_message("Token expired")
sys.exit(0)
# TODO implement this to not crash in parallel
# access_token = await self.ui.send_token_expired()
# self.state_manager.update_access_token(access_token)
# continue
except Exception:
raise APIError("Token expired")
if getattr(err, "status_code", None) == 400 and getattr(err, "message", None) == "not_enough_tokens":
if self.ui:
try:
await self.ui.ask_question(
"",
buttons={},
buttons_only=True,
extra_info={"not_enough_tokens": True},
source=pythagora_source,
)
sys.exit(0)
# TODO implement this to not crash in parallel
# user_response = await self.ui.ask_question(
# 'Not enough tokens left, please top up your account and press "Continue".',
# buttons={"continue": "Continue", "exit": "Exit"},
# buttons_only=True,
# extra_info={"not_enough_tokens": True},
# source=pythagora_source,
# )
# if user_response.button == "continue":
# continue
# else:
# raise APIError("Not enough tokens left")
except Exception:
raise APIError("Not enough tokens left")
try:
if hasattr(err, "response"):
if err.response.headers.get("Content-Type", "").startswith("application/json"):
err_code = err.response.json().get("error", {}).get("code", "")
else:
err_code = str(err.response.text)
elif isinstance(err, str):
err_code = err
else:
err_code = json.dumps(err)
except Exception as e:
err_code = f"Error parsing response: {str(e)}"
err_code = err.response.json().get("error", {}).get("code", "")
if err_code in ("request_too_large", "context_length_exceeded", "string_above_max_length"):
# Handle OpenAI and Groq token limit exceeded
# OpenAI will return `string_above_max_length` for prompts more than 1M characters
@ -363,7 +251,7 @@ class BaseLLMClient:
[
"We sent too large request to the LLM, resulting in an error. ",
"This is usually caused by including framework files in an LLM request. ",
"Here's how you can get Pythagora to ignore those extra files: ",
"Here's how you can get GPT Pilot to ignore those extra files: ",
"https://bit.ly/faq-token-limit-error",
]
)
@ -394,10 +282,7 @@ class BaseLLMClient:
request_log.error = f"Error parsing response: {err}"
request_log.status = LLMRequestStatus.ERROR
log.debug(f"Error parsing LLM response: {err}, asking LLM to retry", exc_info=True)
if response:
convo.assistant(response)
else:
convo.assistant(".")
convo.assistant(response)
convo.user(f"Error parsing response: {err}. Please output your response EXACTLY as requested.")
continue
else:
@ -412,6 +297,19 @@ class BaseLLMClient:
return response, request_log
async def api_check(self) -> bool:
"""
Perform an LLM API check.
:return: True if the check was successful, False otherwise.
"""
convo = Convo()
msg = "This is a connection test. If you can see this, please respond only with 'START' and nothing else."
convo.user(msg)
resp, _log = await self(convo)
return bool(resp)
@staticmethod
def for_provider(provider: LLMProvider) -> type["BaseLLMClient"]:
"""
@ -424,12 +322,9 @@ class BaseLLMClient:
from .azure_client import AzureClient
from .groq_client import GroqClient
from .openai_client import OpenAIClient
from .relace_client import RelaceClient
if provider == LLMProvider.OPENAI:
return OpenAIClient
elif provider == LLMProvider.RELACE:
return RelaceClient
elif provider == LLMProvider.ANTHROPIC:
return AnthropicClient
elif provider == LLMProvider.GROQ:

@ -1,60 +1,9 @@
import json
import re
from enum import Enum
from typing import List, Optional, Union
from typing import Optional, Union
from pydantic import BaseModel, ValidationError, create_model
class CodeBlock(BaseModel):
description: str
content: str
class ParsedBlocks(BaseModel):
original_response: str
blocks: List[CodeBlock]
class DescriptiveCodeBlockParser:
"""
Parse Markdown code blocks with their descriptions from a string.
Returns both the original response and structured data about each block.
Each block entry contains:
- description: The text line immediately preceding the code block
- content: The actual content of the code block
Example usage:
>>> parser = DescriptiveCodeBlockParser()
>>> text = '''file: next.config.js
... ```js
... module.exports = {
... reactStrictMode: true,
... };
... ```'''
>>> result = parser(text)
>>> assert result.blocks[0].description == "file: next.config.js"
"""
def __init__(self):
self.pattern = re.compile(r"^(.*?)\n```([a-z0-9]+\n)?(.*?)^```\s*", re.DOTALL | re.MULTILINE)
def __call__(self, text: str) -> ParsedBlocks:
# Store original response
original_response = text.strip()
# Find all blocks with their preceding text
blocks = []
for match in self.pattern.finditer(text):
description = match.group(1).strip()
content = match.group(3).strip()
# Only add block if we have both description and content
if description and content:
blocks.append(CodeBlock(description=description, content=content))
return ParsedBlocks(original_response=original_response, blocks=blocks)
from pydantic import BaseModel, ValidationError
class MultiCodeBlockParser:
@ -137,7 +86,6 @@ class JSONParser:
def __init__(self, spec: Optional[BaseModel] = None, strict: bool = True):
self.spec = spec
self.strict = strict or (spec is not None)
self.original_response = None
@property
def schema(self):
@ -154,8 +102,7 @@ class JSONParser:
return "\n".join(error_txt)
def __call__(self, text: str) -> Union[BaseModel, dict, None]:
self.original_response = text.strip() # Store the original text
text = self.original_response
text = text.strip()
if text.startswith("```"):
try:
text = CodeBlockParser()(text)
@ -183,17 +130,7 @@ class JSONParser:
except Exception as err:
raise ValueError(f"Error parsing JSON: {err}") from err
# Create a new model that includes the original model fields and the original text
ExtendedModel = create_model(
f"Extended{self.spec.__name__}",
original_response=(str, ...),
**{field_name: (field.annotation, field.default) for field_name, field in self.spec.__fields__.items()},
)
# Instantiate the extended model
extended_model = ExtendedModel(original_response=self.original_response, **model.dict())
return extended_model
return model
class EnumParser:

@ -1,62 +0,0 @@
from typing import Optional
import httpx
from httpx import AsyncClient
from core.config import LLMProvider
from core.llm.base import BaseLLMClient
from core.llm.convo import Convo
from core.log import get_logger
log = get_logger(__name__)
class RelaceClient(BaseLLMClient):
provider = LLMProvider.RELACE
def _init_client(self):
self.headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {self.state_manager.get_access_token() if self.state_manager.get_access_token() is not None else self.config.api_key if self.config.api_key is not None else ''}",
}
self.client = AsyncClient()
async def _make_request(
self,
convo: Convo,
temperature: Optional[float] = None,
json_mode: bool = False,
) -> tuple[str, int, int]:
"""
Make a POST request to the Relace API to merge code snippets.
:param convo: Conversation object containing initial code and edit snippet.
:param temperature: Not used in this implementation.
:param json_mode: Not used in this implementation.
:return: Merged code, input tokens (0), and output tokens (0).
"""
data = {
"initialCode": convo.messages[0]["content"]["initialCode"],
"editSnippet": convo.messages[0]["content"]["editSnippet"],
"model": self.config.model,
}
async with httpx.AsyncClient(transport=httpx.AsyncHTTPTransport()) as client:
try:
response = await client.post(
"https://api.pythagora.io/v1/relace/merge", headers=self.headers, json=data
)
response.raise_for_status()
response_json = response.json()
return (
response_json.get("content", ""),
response_json.get("inputTokens", 0),
response_json.get("outputTokens", 0),
)
except Exception as e:
# Fall back to other ai provider
log.debug(f"Relace API request failed: {e}")
return ("", 0, 0)
__all__ = ["RelaceClient"]

@ -1,65 +1,6 @@
import os
from collections import deque
from logging import FileHandler, Formatter, Logger, StreamHandler, getLogger
from core.config import LogConfig
from core.config.constants import LOGS_LINE_LIMIT
class LineCountLimitedFileHandler(FileHandler):
"""
A file handler that limits the number of lines in the log file.
It keeps a fixed number of the most recent log lines.
"""
def __init__(self, filename, max_lines=LOGS_LINE_LIMIT, mode="a", encoding=None, delay=False):
"""
Initialize the handler with the file and max lines.
:param filename: Log file path
:param max_lines: Maximum number of lines to keep in the file
:param mode: File open mode
:param encoding: File encoding
:param delay: Delay file opening until first emit
"""
super().__init__(filename, mode, encoding, delay)
self.max_lines = max_lines
self.line_buffer = deque(maxlen=max_lines)
self._load_existing_lines()
def _load_existing_lines(self):
"""Load existing lines from the file into the buffer if the file exists."""
if os.path.exists(self.baseFilename):
try:
with open(self.baseFilename, "r", encoding=self.encoding) as f:
for line in f:
if len(self.line_buffer) < self.max_lines:
self.line_buffer.append(line)
else:
self.line_buffer.popleft()
self.line_buffer.append(line)
except Exception:
# If there's an error reading the file, we'll just start with an empty buffer
self.line_buffer.clear()
def emit(self, record):
"""
Emit a record and maintain the line count limit.
:param record: Log record to emit
"""
try:
msg = self.format(record)
line = msg + self.terminator
self.line_buffer.append(line)
# Rewrite the entire file with the current buffer
with open(self.baseFilename, "w", encoding=self.encoding) as f:
f.writelines(self.line_buffer)
self.flush()
except Exception:
self.handleError(record)
def setup(config: LogConfig, force: bool = False):
@ -86,9 +27,7 @@ def setup(config: LogConfig, force: bool = False):
formatter = Formatter(config.format)
if config.output:
# Use our custom handler that limits line count
max_lines = getattr(config, "max_lines", LOGS_LINE_LIMIT)
handler = LineCountLimitedFileHandler(config.output, max_lines=max_lines, encoding="utf-8")
handler = FileHandler(config.output, encoding="utf-8")
else:
handler = StreamHandler()

@ -30,7 +30,6 @@ class LocalProcess:
stdout: str
stderr: str
_process: asyncio.subprocess.Process
show_output: bool
def __hash__(self) -> int:
return hash(self.id)
@ -42,7 +41,6 @@ class LocalProcess:
cwd: str = ".",
env: dict[str, str],
bg: bool = False,
show_output: Optional[bool] = True,
) -> "LocalProcess":
log.debug(f"Starting process: {cmd} (cwd={cwd})")
_process = await asyncio.create_subprocess_shell(
@ -58,7 +56,13 @@ class LocalProcess:
_process.stdin.close()
return LocalProcess(
id=uuid4(), cmd=cmd, cwd=cwd, env=env, stdout="", stderr="", _process=_process, show_output=show_output
id=uuid4(),
cmd=cmd,
cwd=cwd,
env=env,
stdout="",
stderr="",
_process=_process,
)
async def wait(self, timeout: Optional[float] = None) -> int:
@ -86,15 +90,11 @@ class LocalProcess:
:param timeout: Timeout for the read operation (should not be too long).
:return: Data read from the stream reader, or empty string.
"""
buffer = ""
while True:
try:
data = await asyncio.wait_for(reader.read(1), timeout)
if not data:
return buffer
buffer += data.decode("utf-8", errors="ignore")
except asyncio.TimeoutError:
return buffer
try:
data = await asyncio.wait_for(reader.read(), timeout)
return data.decode("utf-8", errors="ignore")
except asyncio.TimeoutError:
return ""
async def read_output(self, timeout: float = NONBLOCK_READ_TIMEOUT) -> tuple[str, str]:
new_stdout = await self._nonblock_read(self._process.stdout, timeout)
@ -186,7 +186,7 @@ class ProcessManager:
for process in procs:
out, err = await process.read_output()
if process.show_output and self.output_handler and (out or err):
if self.output_handler and (out or err):
await self.output_handler(out, err)
if not process.is_running:
@ -206,11 +206,10 @@ class ProcessManager:
cwd: str = ".",
env: Optional[dict[str, str]] = None,
bg: bool = True,
show_output: Optional[bool] = True,
) -> LocalProcess:
env = {**self.default_env, **(env or {})}
abs_cwd = abspath(join(self.root_dir, cwd))
process = await LocalProcess.start(cmd, cwd=abs_cwd, env=env, bg=bg, show_output=show_output)
process = await LocalProcess.start(cmd, cwd=abs_cwd, env=env, bg=bg)
if bg:
self.processes[process.id] = process
return process
@ -222,7 +221,6 @@ class ProcessManager:
cwd: str = ".",
env: Optional[dict[str, str]] = None,
timeout: float = MAX_COMMAND_TIMEOUT,
show_output: Optional[bool] = True,
) -> tuple[Optional[int], str, str]:
"""
Run command and wait for it to finish.
@ -234,7 +232,6 @@ class ProcessManager:
:param cwd: Working directory.
:param env: Environment variables.
:param timeout: Timeout in seconds.
:param show_output: Show output in the ui.
:return: Tuple of (status code, stdout, stderr).
"""
timeout = min(timeout, MAX_COMMAND_TIMEOUT)
@ -244,7 +241,7 @@ class ProcessManager:
t0 = time.time()
while process.is_running and (time.time() - t0) < timeout:
out, err = await process.read_output(BUSY_WAIT_INTERVAL)
if self.output_handler and (out or err) and show_output:
if self.output_handler and (out or err):
await self.output_handler(out, err)
if process.is_running:
@ -255,7 +252,7 @@ class ProcessManager:
await process.wait()
out, err = await process.read_output()
if self.output_handler and (out or err) and show_output:
if self.output_handler and (out or err):
await self.output_handler(out, err)
if terminated:

@ -1,10 +0,0 @@
You're starting a new software project. The specification provided by the client is:
```
{{ project_description }}
```
Based on the specification, we've decided to use the following project scaffolding/template: {{ project_template.description }}.
To start, we need to specify options for the project template:
{{ project_template.options_description }}

@ -1,33 +0,0 @@
You're designing the architecture and technical specifications for a new project.
To speed up the project development, you need to consider if you should use a project template or start from scratch. If you decide to use a template, you should choose the one that best fits the project requirements.
Here is a high level description of "{{ state.branch.project.name }}":
```
{{ state.specification.description }}
```
You have an option to use project templates that implement standard boilerplate/scaffolding so you can start faster and be more productive. To be considered, a template must be compatible with the project requirements:
* if the project description has specific technology requirements, don't consider templates that choose different tech (eg. a different framework or library)
* to be considered, the template must use compatible technologies and implement a useful subset of required functionality
If no project templates are a good match, don't pick any! It's better to start from scratch than to use a template that is not a good fit for the project (for example, don't use a react frontend if a different framework or plain html/css is required) and then spend time reworking it to fit the requirements. If you do choose to pick a template, choose the one that's the best match for this project.
Here are the available project templates:
{% for template in templates.values() %}
### {{ template.name }} ({{ template.stack }})
{{ template.description }}
Contains:
{{ template.summary }}
{% endfor %}
Output your response in a valid JSON format like in this example:
```json
{
"architecture": "Detailed description of the architecture of the application",
"template": "foo" // or null if you decide not to use a project template
}
```

@ -2,19 +2,33 @@ You're designing the architecture and technical specifications for a new project
If the project requirements call out for specific technology, use that. Otherwise, if working on a web app, prefer Node.js for the backend (with Express if a web server is needed, and MongoDB if a database is needed), and Bootstrap for the front-end. You MUST NOT use Docker, Kubernetes, microservices and single-page app frameworks like React, Next.js, Angular, Vue or Svelte unless the project details explicitly require it.
Here is a high level description of "{{ state.branch.project.name }}":
```
{{ state.specification.description }}
```
Here are the details for the new project:
-----------------------------
{% include "partials/project_details.prompt" %}
{% include "partials/features_list.prompt" %}
-----------------------------
Based on these details, think step by step to design the architecture for the project and choose technologies to use in building it.
Here is a short description of the project architecture:
{{ architecture }}
1. First, design and describe project architecture in general terms
2. Then, list any system dependencies that should be installed on the system prior to start of development. For each system depedency, output a {{ os }} command to check whether it's installed.
3. Finally, list any other 3rd party packages or libraries that will be used (that will be installed later using packager a package manager in the project repository/environment).
4. {% if templates %}Optionally, choose a project starter template.{% else %}(for this project there are no available starter/boilerplate templates, so there's no template to choose){% endif %}
Based on these details, think step by step and choose technologies to use in building it.
{% if templates %}
You have an option to use a project template that implements standard boilerplate/scaffolding so you can start faster and be more productive. To be considered, a template must be compatible with the architecture and technologies you've choosen (it doesn't need to implement everything that will be used in the project, just a useful subset). If multiple templates can be considered, pick one that's the best match.
1. First, list any system dependencies that should be installed on the system prior to start of development. For each system depedency, output a {{ os }} command to check whether it's installed.
2. Then, list any other 3rd party packages or libraries that will be used (that will be installed later using packager a package manager in the project repository/environment).
3. Finally, list the folder structure of the project, including any key files that should be included.
If no project templates are a good match, don't pick any! It's better to start from scratch than to use a template that is not a good fit for the project and then spend time reworking it to fit the requirements.
Here are the available project templates:
{% for name, tpl in templates.items() %}
### {{ name }}
{{ tpl.description }}
Contains:
{{ tpl.summary }}
{% endfor %}
{% endif %}
*IMPORTANT*: You must follow these rules while creating your project:
@ -26,6 +40,7 @@ Based on these details, think step by step and choose technologies to use in bui
Output only your response in JSON format like in this example, without other commentary:
```json
{
"architecture": "Detailed description of the architecture of the application",
"system_dependencies": [
{
"name": "Node.js",
@ -47,6 +62,7 @@ Output only your response in JSON format like in this example, without other com
"description": "Express web server for Node"
},
...
]
],
"template": "name of the project template to use" // or null if you decide not to use a project template
}
```

@ -1,4 +0,0 @@
The developer wants to ask you a question. Here is the question:
{{question}}
Please answer and refer to all the files in the repository and everything we've talked about so far but do not form your answer in any way that was asked for before, just answer the question as if you're talking to a colleague.

@ -1,6 +0,0 @@
We are working on a solving a technical problem in a codebase and here is a conclusion from a team member:
--- TEAM_MEMBER_CONCLUSION ---
{{ hunt_conclusion }}
--- END_OF_TEAM_MEMBER_CONCLUSION ---
Please tell me if the conclusion from the team member is to add more logs around the code or if the conclusion is that there are all information needed to fix the issue.

@ -1,7 +0,0 @@
Tell me the most important logs that are relevant for this issue. For each log, tell me the the following:
1. line in the code (eg. `print(...)`, `console.log(...)`, etc.) that generated the log
2. what file is the log in (eg. `index.js`, `routes/users.js`, etc. - make sure to put the entire path like listed above)
2. the current output of that log (make sure not to put the entire log output but maximum 5-10 lines of the output)
3. the expected output of that log (also make sure to put maximum of 5-10 lines of the output)
4. should the log be different from the current output or are the current and expected output the same
5. a brief explanation of why the output is incorrect and what should be different here (use maximum 2-3 sentences)

@ -1,30 +0,0 @@
You are working on an app called "{{ state.branch.project.name }}" and you need to write code for the entire application.
{% include "partials/project_details.prompt" %}
{% if state.tasks and state.current_task %}
Development process of this app was split into smaller tasks. Here is the list of all tasks:
```{% for task in state.tasks %}
{{ loop.index }}. {{ task.description }}
{% endfor %}
```
You are currently working on, and have to focus only on, this task:
```
{{ current_task.description }}
```
{% endif %}
A part of the app is already finished.
{% include "partials/files_list.prompt" %}
{% include "partials/user_feedback.prompt" %}
{% if next_solution_to_try is not none %}
Focus on solving this issue in the following way:
```
{{ next_solution_to_try }}
```
{% endif %}
Based on this information, you need to tell me in 2-3 sentences how can I reproduce the issue that the user experienced.

@ -1,6 +0,0 @@
The human is sending you a hint about how to solve this bug. Here is what human said:
```
{{ human_hint }}
```
Now, based on this hint, break down exactly what the problem is, what is the solution to this problem and how can we implement this solution so that the bug is fixed.

@ -1,48 +0,0 @@
You are working on an app called "{{ state.branch.project.name }}" and you need to write code for the entire application.
{% include "partials/project_details.prompt" %}
{% if state.tasks and state.current_task %}
Development process of this app was split into smaller tasks. Here is the list of all tasks:
```{% for task in state.tasks %}
{{ loop.index }}. {{ task.description }}
{% endfor %}
```
You are currently working on, and have to focus only on, this task:
```
{{ current_task.description }}
```
{% endif %}
A part of the app is already finished.
{% include "partials/files_list.prompt" %}
{% include "partials/user_feedback.prompt" %}
{% if test_instructions %}
Here are the test instructions the user was following when the issue occurred:
```
{% for step in test_instructions %}
Step #{{ loop.index }}
Action: {{ step.action }}
Expected result: {{ step.result }}
{% endfor %}
```
{% endif %}
{% if next_solution_to_try is not none %}
Focus on solving this issue in the following way:
```
{{ next_solution_to_try }}
```
{% endif %}
Based on this information, you need to figure out where is the problem that the user described. You have 2 options - to tell me exactly where is the problem happening or to add more logs to better determine where is the problem.
If you think we should add more logs around the code to better understand the problem, tell me code snippets in which we should add the logs. If you think you know where the issue is, don't add any new logs but explain what log print tell point you to the problem, what the problem is, what is the solution to this problem and how the solution will fix the problem. What is your answer?
**IMPORTANT**
If you want code to be written, write **ALL NEW CODE** that needs to be written. If you want to create a new file, write the entire content of that file and if you want to update an existing file, write the new code that needs to be written/updated. You cannot answer with "Ensure that...", "Make sure that...", etc. In these cases, explain how should the reader of your message ensure what you want them to ensure. In most cases, they will need to add some logs to ensure something in which case tell them where to add them.
** IMPORTANT - labels around code **
Always address code that needs to be changed by files and add labels <pythagoracode file="client/src/api/api.ts"> and </pythagoracode> around changes for a specific file. (in this case client/src/api/api.ts) - you can mention multiple changes for a single file but never mix changes for multiple files in a single <pythagoracode> block. Never use any other markers around the code like backticks.

@ -1,25 +0,0 @@
{% if backend_logs and backend_logs|trim %}
Here are the logs we added to the backend:
```
{{ backend_logs }}
```
{% endif %}{% if frontend_logs and frontend_logs|trim %}
Here are the logs we added to the frontend:
```
{{ frontend_logs }}
```
{% endif %}{% if user_feedback and user_feedback|trim %}
Finally, here is a hint from a human who tested the app:
```
{{ user_feedback }}
```
When you're thinking about what to do next, take into the account human's feedback.
{% endif %}{% if fix_attempted %}
The problem wasn't solved with the last changes. You have 2 options - to tell me exactly where is the problem happening or to add more logs to better determine where is the problem. If you think we should add more logs around the code to better understand the problem, tell me code snippets in which we should add the logs. If you think you know where the issue is, don't add any new logs but explain what log print tell point you to the problem, what the problem is, what is the solution to this problem and how the solution will fix the problem. What is your answer? Make sure not to repeat mistakes from before that didn't work.
{% endif %}
{% if not (backend_logs and backend_logs|trim) and
not (frontend_logs and frontend_logs|trim) and
not (user_feedback and user_feedback|trim) and
not fix_attempted %}
Human didn't supply any data
{% endif %}

@ -1,4 +0,0 @@
This also didn't help to solve the issue so we can conclude that you are unable to solve this problem yourself so I got a human here who will help you out.
First, focus on the problem you're facing and explain it to the human. Explain what is the issue that you're working in and what should the human try to do to solve this problem. Is there anything the human can look at that you don't have access to - a database, API response, etc.? If there is something for the human to look at, specify exactly how can the human obtain this information. Keep the answer short and to the point.

@ -1 +0,0 @@
Please tell me more about the problem we're working on and don't repeat things you said before but tell me something I don't know.

@ -1,35 +0,0 @@
This is the description of the app you are working with:
{{ initial_description }}
{% if task_description is defined and task_description %}
Currently, you are working on the following task:
{{ task_description }}
{% endif %}
{% if bug_hunt_cycle_user_feedback is defined and bug_hunt_cycle_user_feedback %}
You are currently working on a bug report that the user reported like this:
{{ bug_hunt_cycle_user_feedback }}
{% endif %}
{% if testing_instructions is defined and testing_instructions %}
The user is currently testing how the task was implemented and was given the following instructions to test the app:
{{ testing_instructions }}
{% endif %}
{% if command_run is defined and command_run %}
Currently, you're waiting for the user to approve the following command to be run:
```text
{{ command_run }}
```
{% endif %}
{% if human_intervention is defined and human_intervention %}
Currently, you asked the human user to do the following steps in order to continue with the development of the app:
```text
{{ human_intervention }}
```
{% endif %}
{% if user_input is defined and user_input %}
Now, the human asked said the following: `{{ user_input }}`. Please respond as a professional developer, be helpful and focus on what the human said.
{% endif %}

@ -1,5 +0,0 @@
You are a world class full stack software developer working in a team.
You write modular, well-organized code split across files that are not too big, so that the codebase is maintainable. You include proper error handling and logging for your clean, readable, production-level quality code.
Your job is to implement tasks assigned by your tech lead, following task implementation instructions.

@ -2,7 +2,7 @@ Your task is to explain the functionality implemented by a particular source cod
Given a file path and file contents, your output should contain:
* a detailed explanation of what the file is about, max 30 words;
* a detailed explanation of what the file is about;
* a list of all other files referenced (imported) from this file. note that some libraries, frameworks or libraries assume file extension and don't use it explicitly. For example, "import foo" in Python references "foo.py" without specifying the extension. In your response, use the complete file name including the implied extension (for example "foo.py", not just "foo").
Please analyze file `{{ path }}`, which contains the following content:

@ -1,20 +1,22 @@
{% if file_content %}
You are working on a project and your job is to implement new code changes based on given instructions.
Now you have to implement ALL changes that are related to `{{ file_name }}` described in development instructions listed below.
Make sure you don't make any mistakes, especially ones that could affect rest of project. Your changes will be reviewed by very detailed reviewer. Because of that, it is extremely important that you are STRICTLY following ALL the following rules while implementing changes:
{% else %}
You are working on a project and your job is to create a new file `{{ file_name }}` based on given instructions. The file should be thoroughly described in the development instructions listed below. You need to follow the coding rules that will be listed below, read the development instructions and respond with the full contents of the file `{{ file_name }}`.
{% endif %}
{% include "partials/coding_rules.prompt" %}
You are currently working on this task:
```
{{ state.current_task.description }}
```
{% include "partials/user_feedback.prompt" %}
Here are development instructions that were sent to you by a senior developer that you need to carefully follow. Focus only on the code changes for the file `{{ file_name }}`:
~~~START_OF_DEVELOPMENT_INSTRUCTIONS~~~
Here are development instructions and now you have to focus only on changes in `{{ file_name }}`:
---start_of_development_instructions---
{{ instructions }}
~~~END_OF_DEVELOPMENT_INSTRUCTIONS~~~
---end_of_development_instructions---
{% if rework_feedback is defined %}
You previously made changes to file `{{ file_name }}` but not all changes were accepted, and the reviewer provided feedback on the changes that you must rework:
@ -26,15 +28,12 @@ The reviewer accepted some of your changes, and the file now looks like this:
{{ file_content }}
```
{% elif file_content %}
Now, take a look at how `{{ file_name }}` looks like currently:
Here is how `{{ file_name }}` looks like currently:
```
{{ file_content }}
```
Ok, now, you have to follow the instructions about `{{ file_name }}` from the development instructions carefully. Reply **ONLY** with the full contents of the file `{{ file_name }}` and nothing else. Do not make any changes to the file that are not mentioned in the development instructions - you must **STRICTLY** follow the instructions.
{% else %}
You need to create a new file `{{ file_name }}` so respond **ONLY** with the full contents of that file from the development instructions that you read.
You need to create a new file `{{ file_name }}`.
{% endif %}
** IMPORTANT **
Remember, you must **NOT** add anything in your response that is not strictly the code from the file. Do not start or end the response with an explanation or a comment - you must respond with only the code from the file because your response will be directly saved to a file and run.
{% include "partials/files_list.prompt" %}

@ -1 +0,0 @@
{% extends "troubleshooter/iteration.prompt" %}

@ -0,0 +1,2 @@
{# This is the same template as for Developer's breakdown because Code Reviewer is reusing it in a conversation #}
{% extends "developer/breakdown.prompt" %}

@ -18,7 +18,7 @@ As you can see, there {% if hunks|length == 1 %}is only one hunk in this diff, a
When reviewing the code changes, apply these principles to decide on each hunk:
- Apply: Approve and integrate the hunk into our core codebase if it accurately delivers the intended functionality or enhancement, aligning with our project objectives. This action confirms the change is beneficial and meets our quality standards.
- Ignore: Use this option sparingly, only when you're certain the entire hunk is incorrect or will introduce errors (logical, syntax, etc.) that could negatively impact the project. Ignoring means the hunk will be completely removed. This should be reserved for cases where the inclusion of the code is definitively more harmful than its absence. Emphasize careful consideration before choosing 'Ignore.' It's crucial for situations where the hunk's removal is the only option to prevent significant issues. Otherwise, 'Rework' might be the better choice to ensure the code's integrity and functionality.
- Rework: Suggest this option if the concept behind the change is valid and necessary but is implemented in a way that introduces problems. This indicates a need for a revision of the hunk to refine its integration without fully discarding the underlying idea. DO NOT suggest making changes to files other than the one you're reviewing.
- Rework: Suggest this option if the concept behind the change is valid and necessary but is implemented in a way that introduces problems. This indicates a need for a revision of the hunk to refine its integration without fully discarding the underlying idea.
When deciding what should be done with the hunk you are currently reviewing, pick an option that most reviewers of your skill would choose. Your decisions have to be consistent.

@ -0,0 +1,2 @@
You are a world class full stack software developer. You write modular, clean, maintainable, production-ready code.
Your job is to review changes implemented by your junior team members.

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save