Documentation and some extensions to make it work in a GPT

.github/workflows/master.yml

@@ -59,7 +59,7 @@ jobs:
     #  run: $MVNCMD org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce -Drules=requireReleaseVersion,requireReleaseDeps
 
     - name: Check build with Maven
-      run: $MVNCMD verify
+      run: $MVNCMD clean verify
 
     - name: Site with Maven
       run: $MVNCMD install javadoc:aggregate site site:stage

README.md

@@ -1,115 +1,18 @@
 # Developers ChatGPT ToolBench
 
-This is a [ChatGPT plugin](https://chat.openai.com) for developers that allows reading / searching / writing files
-in the local directory it is started in, and to execute configured actions, e.g. a build and test run.
+Would you like to have ChatGPT list, search, read and modify your local files and have it execute (e.g. build and test)
+actions locally to support you in your development processes? Then this might be for you. The Developers ChatGPT
+ToolBench can either work as a [ChatGPT plugin](https://openai.com/blog/chatgpt-plugins) or provide the actions for a
+OpenAI
+[GPT](https://openai.com/blog/introducing-gpts) for read or even write access to the files in the local directory it is
+started in.
+
 In contrast to other approaches like [AutoGPT](https://github.com/Significant-Gravitas/AutoGPT) this is not meant to
-autonomously execute extensive changes - which would require a lot of prompt engineering, but to enable the
+autonomously execute extensive changes (which would likely require a lot of prompt engineering), but to enable the
 developer to flexibly use the AI within a ChatGPT chat session for various tasks both to analyze code and to make
 some changes.
 
-The plugin is quite stable and useable. In fact I use it regularily in my own development - both on itself and on
-other projects. Using it does, however, require that you have a paid ChatGPT account that can use plugins.
-Also, the plugin runs locally on your machine to be able to read and write the files in it's working directory.
-To access a local plugin you have to register yourself as a [plugin developer](https://openai.com/waitlist/plugins).
-
-## Purpose
-
-The `Developers ChatGPT ToolBench` is a Java application designed to provide a plugin for ChatGPT that allows the AI to
-access, read, and write files in the directory where the plugin is started. The plugin is implemented as an
-executable jar. If you check out and compile this Git repository, you can also use the script
-[bin/developersToolBenchPlugin](bin/developersToolBenchPlugin) after building it with
-[bin/developersToolBenchPlugin-buildStable](bin/developersToolBenchPlugin-buildStable).
-
-The plugin provides several operations, including:
-
-- Listing the files in a directory
-- Reading the contents of a file
-- Writing content to a file / changing file content
-- searching for regular expressions in files
-- Executing a shell script with given content as standard input
-- fetch the text content of an URL
-
-Caution: you currently need to be a paying user of ChatGPT (to have access to ChatGPT-4) and to be registered for the
-ChatGPT Plugin beta, probably as a plugin developer.
-
-## Status
-
-I'm regularily use it for my own development, and it seems quite stable. I'm using it quite often when I'm asking
-ChatGPT questions or small extensions.
-
-## Usage
-
-To use the `Developers ChatGPT ToolBench`, you need to have registered as a plugin developer with ChatGPT.
-Once you've done that, you can add the `Developers ChatGPT ToolBench` using the
-"Develop your own plugin" option in the ChatGPT web interface with URL "localhost:3002". (You could also specify
-another port on the command line when starting it, if you like.)
-
-To start the plugin, navigate to the directory you want to access and run the `bin/developersToolBenchPlugin` script.
-The plugin will start a server on port 3002 (by default) and will be ready to accept requests from ChatGPT. If you
-want to give a port use option -p (portnumber) ; if you want to write files add option -w .
-
-The plugin is written so that it cannot be used to go outside of the directory ("../somefile" won't work) and also
-files starting with a dot or containg /target/ are invisible and not writeable. That prevents directories like .git
-to be touched and maven target folders tend to contain very much stuff.
-(Compare regex IGNORE_FILE in the script).
-
-Use the writing features at your own risk. There will likely be problems with large files. A possible approach is
-have ChatGPT write things, but frequently make a git commit to easily inspect changes and be able to revert.
-
-If there is a file named .cgptdevbench/.requestlog.txt the requests are logged into that, to see what ChatGPT did.
-
-## Download
-
-You can run it from the source (build the program with bin/developersToolbenchPlugin-buildStable) and run it with
-bin/developersToolbenchPlugin, or [download a release](https://github.com/stoerr/DevelopersChatGPTToolBench/releases)
-and run the executable jar in whatever directory you want to access. You could use the script
-[bin/developersToolbenchPlugin](bin/developersToolbenchPlugin) as an example how to run that downloaded jar.
-
-## Examples
-
-Here are some examples of how to use the `Developers ChatGPT ToolBench`:
-
-- **List Files**: To list the files in the current directory, you can use the `listFiles` operation. In ChatGPT, you
-  would ask the AI to list the files in the directory, and it would send a request to the plugin to perform this
-  operation. It currently lists all files recursively, so don't use a too large directory.
-
-- **Read File**: To read the contents of a file, you can use the `readFile` operation. In ChatGPT, you would ask the AI
-  to read a specific file, and it would send a request to the plugin to perform this operation.
-
-- **Write File**: To write content to a file, you can use the `writeFile` operation. In ChatGPT, you would ask the AI to
-  write a specific content to a file, and it would send a request to the plugin to perform this operation.
-
-- **Search Files**: to search for Strings in files, you can use the 'grepFiles' operation. In ChatGPT, you could ask
-  to search for files with a file name pattern and containing a string or pattern.
-
-- **Execute Action**: To execute a shell script with given content as standard input, you can use the `executeAction`
-  operation. In ChatGPT, you would ask the AI to execute a specific action, and it would send a request to the plugin to
-  perform this operation. The shell script should be located at `.cgptfmgr/{actionName}.sh`, where `{actionName}` is a
-  parameter provided in the query string. The content is passed as standard input to the shell script. Some examples
-  are in [.cgptdevbench/](.cgptdevbench/) and
-  [src/test/resources/testdir/.cgptdevbench](src/test/resources/testdir/.cgptdevbench) .
-
-- **Fetch the text content of an URL**: gives ChatGPT simple web access: this can perform a GET request and returns
-  the text content of the URL (not the HTML) to ChatGPT.
-
-Remember, the `Developers ChatGPT ToolBench` operates on the directory where it was started,
-so be careful to start it in a directory that contains the files you want to access.
-
-## Configuring FileManagerPlugin for use in ChatGPT
-
-To use the `Developers ChatGPT ToolBench` with ChatGPT, you need to register it as a plugin in the ChatGPT interface.
-Here's a step-by-step guide on how to do this:
-
-1. **Register as a Plugin Developer**: If you haven't already, register as a plugin developer with ChatGPT. This will
-   give you access to the plugin developer interface where you can add your own plugins.
-
-2. **Start the Plugin**: Navigate to the directory you want to access and run the `Developers ChatGPT ToolBench`
-   program. This will start a server on port 3002 (by default).
-
-3. **Add the Plugin**: In the ChatGPT interface, navigate to the plugin developer section and select "Develop your own
-   plugin", and enter the url `localhost:3002`
+The project is quite stable and useable. In fact I use it regularily in my own development - both on itself and on
+my other projects. Using it does, however, require that you have a paid ChatGPT account that can use plugins / GPTs.
 
-4. **Test the Plugin**: Once you've added the plugin, you can test it in the ChatGPT interface. Try asking the AI to
-   list the files in the directory, read a specific file, write to a file, or execute a specific action after
-   setting up some actions in a .cgptdevbench directory in the directory you're running it. If everything
-   is set up correctly, the AI should be able to perform these operations using the plugin.
+For more information see the [documentation site](https://stoerr.github.io/DevelopersChatGPTToolBench/).

_includes/head-custom.html

@@ -1,5 +1,5 @@
 <link rel="shortcut icon" type="image/x-icon" href="/DevelopersChatGPTToolBench/favicon.ico">
-<!-- Google tag (gtag.js) -->
+<!-- Google tag (gtag.js) for hans-peter stoerr's site -->
 <script async src="https://www.googletagmanager.com/gtag/js?id=G-SL01MYCNDC"></script>
 <script>
   window.dataLayer = window.dataLayer || [];

bin/developersToolbenchPlugin

@@ -1,8 +1,20 @@
 #!/usr/bin/env bash
+# start the developers-chatgpt-toolbench-plugin.jar in the directory this script is placed, following links
 progfile=$0
 if test -L "$progfile"; then
   progfile=$(readlink "$progfile")
 fi
-progdir=$(dirname "$progfile")/..
+progdir=$(dirname "$progfile")
 
-/usr/bin/java -jar "$progdir/bin/developers-chatgpt-toolbench-plugin.jar" "$@"
+# if there is a ~/.cgptdevbenchglobal/tunnel.sh then run it and kill it when this script exits
+if test -f ~/.cgptdevbenchglobal/tunnel.sh; then
+  echo "Starting tunnel"
+  ~/.cgptdevbenchglobal/tunnel.sh &
+  tunnelpid=$!
+  trap "echo 'Killing tunnel $tunnelpid'; kill $tunnelpid" EXIT
+  echo "Tunnel pid: $tunnelpid"
+  sleep 1
+  echo
+fi
+
+exec java -jar "$progdir/developers-chatgpt-toolbench-plugin.jar" "$@"

bin/developersToolbenchPlugin-buildStable

@@ -7,7 +7,8 @@ if test -L "$progfile"; then
 fi
 progdir=$(dirname "$progfile")/..
 cd "$progdir" || exit 1
-mvn -B clean install || exit 1
+mvn -B clean test || exit 1
+mvn -B -Dmaven.test.skip.exec -DskipITs -Ddocker.skip.run install || exit 1
 
 echo Copying to bin
 mv -f bin/developers-chatgpt-toolbench-plugin.jar bin/developers-chatgpt-toolbench-plugin.bak.jar

bin/sitebuildloop.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+# Periodically Checks whether there is any file in src/site and it's subdirectories that is newer than target/site/index.html
+# or whether there is no target/site/index.html at all.
+# If so, it runs mvn -N clean site
+
+while true;
+do
+    if [ ! -f target/site/index.html ]; then
+        mvn -N clean site
+    fi
+
+    if [ `find src/site -newer target/site/index.html | wc -l` -gt 0 ]; then
+        mvn -N clean site
+    fi
+
+    sleep 10
+done

pom.xml

@@ -28,8 +28,8 @@
 
     <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <maven.compiler.source>16</maven.compiler.source>
-        <maven.compiler.target>16</maven.compiler.target>
+        <maven.compiler.source>1.8</maven.compiler.source>
+        <maven.compiler.target>11</maven.compiler.target>
         <mainClass>net.stoerr.chatgpt.devtoolbench.DevToolBench</mainClass>
     </properties>
 
@@ -135,18 +135,46 @@
         </resources>
 
         <plugins>
+            <plugin>
+              <groupId>org.apache.maven.plugins</groupId>
+              <artifactId>maven-enforcer-plugin</artifactId>
+              <version>3.4.1</version>
+              <executions>
+                <execution>
+                  <id>enforce-maven</id>
+                  <goals>
+                    <goal>enforce</goal>
+                  </goals>
+                  <configuration>
+                    <rules>
+                      <requireMavenVersion>
+                        <version>3.3.9</version>
+                      </requireMavenVersion>
+                    </rules>
+                  </configuration>
+                </execution>
+              </executions>
+            </plugin>
+
             <!-- Reports -->
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-site-plugin</artifactId>
-                <version>3.12.1</version>
+                <version>4.0.0-M12</version>
                 <configuration>
                     <markdownToHtml>
                         <doxiaModuleNames>
                             <doxiaModuleName>markdown</doxiaModuleName>
                         </doxiaModuleNames>
                     </markdownToHtml>
                 </configuration>
+                <dependencies>
+                    <dependency>
+                        <groupId>io.github.devacfr.maven.skins</groupId>
+                        <artifactId>reflow-velocity-tools</artifactId>
+                        <version>2.3.4</version>
+                    </dependency>
+                </dependencies>
             </plugin>
 
             <plugin>
@@ -185,7 +213,7 @@
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-failsafe-plugin</artifactId>
-                <version>3.2.2</version>
+                <version>3.2.3</version>
                 <executions>
                     <execution>
                         <goals>
@@ -199,7 +227,7 @@
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-surefire-report-plugin</artifactId>
-                <version>3.2.2</version>
+                <version>3.2.3</version>
             </plugin>
             <plugin>
                 <groupId>org.jacoco</groupId>
@@ -209,7 +237,7 @@
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-javadoc-plugin</artifactId>
-                <version>3.6.2</version>
+                <version>3.6.3</version>
             </plugin>
             <plugin>
                 <groupId>org.codehaus.mojo</groupId>

src/main/java/net/stoerr/chatgpt/devtoolbench/AbstractPluginAction.java

@@ -1,5 +1,6 @@
 package net.stoerr.chatgpt.devtoolbench;
 
+import static java.util.stream.Collectors.toList;
 import static net.stoerr.chatgpt.devtoolbench.TbUtils.logBody;
 import static net.stoerr.chatgpt.devtoolbench.TbUtils.logInfo;
 
@@ -15,6 +16,7 @@
 import java.util.Locale;
 import java.util.Map;
 import java.util.regex.Pattern;
+import java.util.stream.Collectors;
 import java.util.stream.Stream;
 
 import javax.annotation.Nullable;
@@ -51,7 +53,7 @@ protected static ExecutionAbortedException sendError(HttpServletResponse respons
     protected static Stream<Path> findMatchingFiles(HttpServletResponse response, Path path, Pattern filePathPattern, Pattern grepPattern) {
         List<Path> result = new ArrayList<>();
         try {
-            Files.walkFileTree(path, new SimpleFileVisitor<>() {
+            Files.walkFileTree(path, new SimpleFileVisitor<Path>() {
                 @Override
                 public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
                     if (isIgnored(dir)) {
@@ -129,15 +131,16 @@ protected Path getPath(HttpServletRequest request, HttpServletResponse response,
         if (mustExist && !Files.exists(resolved)) {
             String message = "Path " + path + " does not exist! Try to list files with /listFiles to find the right path.";
             String filename = resolved.getFileName().toString();
-            List<Path> matchingFiles = findMatchingFiles(response, DevToolBench.currentDir, null, null).toList();
+            List<Path> matchingFiles = findMatchingFiles(response, DevToolBench.currentDir, null, null)
+                    .collect(toList());
             List<String> files = matchingFiles.stream()
                     .map(p -> DevToolBench.currentDir.relativize(p).toString())
                     .map(p -> Pair.of(p, StringUtils.getFuzzyDistance(p, filename, Locale.getDefault())))
                     .map(p -> Pair.of(p.getLeft(), -p.getRight()))
                     .sorted(Comparator.comparingDouble(Pair::getRight))
                     .limit(10)
                     .map(Pair::getLeft)
-                    .toList();
+                    .collect(toList());
             if (!files.isEmpty()) {
                 message += "\n\nDid you mean one of these files?\n" + String.join("\n", files);
                 if (files.size() < matchingFiles.size()) {