I’ve been building a tool that uses ChatGPT behind the scenes and have found that that’s just part of the process of building a prompt and getting the results you want. It also depends on which chat model is being used. If you’re super vague, it’s going to give you rubbish every time. If you go back and forth with it though, you can keep whittling it down to give you better material. If you’re generating content, you can even tell it what format and structure to give the information back in (I learned how to make it give me JSON and markdown only).
Additionally, you can give ChatGPT a description of what it’s role is alongside the prompt, if you’re using the API and have control of that kind of thing. I’ve found that can help shape the responses up nicely right out of the box.
ChatGPT is very, very much a “your mileage may vary” tool. It needs to be setup well at the start, but so many companies have haphazardly jumped on using it and they haven’t put in enough work prepping it.
Have you see the JollyRoger Telco - they’ve started using ChatGPT to help have longer conversations with telemarketing scammers. I might actually re-subscribe to the jolly roger (used them previously) if the new updated bots perform as well enough.
If you don’t mind me asking, does your tool programmatically do the “whittling down” process by talking to ChatGPT behind the scenes, or does the user still talk to it directly? The former seems like a powerful technique, though tricky to pull off in practice, so I’m curious if anyone has managed it.
Don’t mind at all! Yeah, it does a ton of the work behind the scenes. I essentially have a prompt I spent quite a bit of time iterating on. Then from there, what the user types gets sent bundled in with my prompt bootstrap. So it reduces the work considerably for the user and dials it in.
Edit: adding some more context/opinions.
I think the error that a lot of tools make is that they don’t spend enough time shaping their instructions for the AI. Sure, you can offload a lot of the work to it, but you have to write your own guard rails and instructions. You can tell it things like you would a human, and it will sometimes even fill in the gaps.
For example, I asked it to give me a data structure back that included an optional “title”. I found that if you left the title blank, ChatGPT took it upon itself to generate a title for you based on the content it wrote.
A lot of the things I got it to do took time and a ton of test iterations. I was even able to give it a list of exactly how it should structure the content it gave back. Things that I would otherwise do on the programming side, I was able to simply instruct ChatGPT to handle instead.
Ah, interesting. I myself have made my own library to create callable “prompt functions” that prompt the model and validate the JSON outputs, which ensures type-safety and easy integration with normal code.
Lately, I’ve shifted more towards transforming ChatGPT’s outputs. By orchestrating multiple prompts and adding human influence, I can obtain responses that ChatGPT alone likely wouldn’t have come up with. Though, this has to be balanced with giving it the freedom to pursue a different thought process.
What method did you use to generate only JSON? I’m using it (gpt3.5-turbo) in a prototype application, and even with giving it an example (one-shot prompting) and telling it to only output JSON, it sometimes gives me invalid results. I’ve read that the new function-calling feature is still not guaranteed to produce valid json. Microsoft’s “guidance” (https://github.com/microsoft/guidance) looks like what I need, but I haven’t got around to trying it yet.
I’ve been building a tool that uses ChatGPT behind the scenes and have found that that’s just part of the process of building a prompt and getting the results you want. It also depends on which chat model is being used. If you’re super vague, it’s going to give you rubbish every time. If you go back and forth with it though, you can keep whittling it down to give you better material. If you’re generating content, you can even tell it what format and structure to give the information back in (I learned how to make it give me JSON and markdown only).
Additionally, you can give ChatGPT a description of what it’s role is alongside the prompt, if you’re using the API and have control of that kind of thing. I’ve found that can help shape the responses up nicely right out of the box.
ChatGPT is very, very much a “your mileage may vary” tool. It needs to be setup well at the start, but so many companies have haphazardly jumped on using it and they haven’t put in enough work prepping it.
Have you see the JollyRoger Telco - they’ve started using ChatGPT to help have longer conversations with telemarketing scammers. I might actually re-subscribe to the jolly roger (used them previously) if the new updated bots perform as well enough.
Lol that is brilliant use of it. I’ll have to check that out.
If you don’t mind me asking, does your tool programmatically do the “whittling down” process by talking to ChatGPT behind the scenes, or does the user still talk to it directly? The former seems like a powerful technique, though tricky to pull off in practice, so I’m curious if anyone has managed it.
Don’t mind at all! Yeah, it does a ton of the work behind the scenes. I essentially have a prompt I spent quite a bit of time iterating on. Then from there, what the user types gets sent bundled in with my prompt bootstrap. So it reduces the work considerably for the user and dials it in.
Edit: adding some more context/opinions.
I think the error that a lot of tools make is that they don’t spend enough time shaping their instructions for the AI. Sure, you can offload a lot of the work to it, but you have to write your own guard rails and instructions. You can tell it things like you would a human, and it will sometimes even fill in the gaps.
For example, I asked it to give me a data structure back that included an optional “title”. I found that if you left the title blank, ChatGPT took it upon itself to generate a title for you based on the content it wrote.
A lot of the things I got it to do took time and a ton of test iterations. I was even able to give it a list of exactly how it should structure the content it gave back. Things that I would otherwise do on the programming side, I was able to simply instruct ChatGPT to handle instead.
Ah, interesting. I myself have made my own library to create callable “prompt functions” that prompt the model and validate the JSON outputs, which ensures type-safety and easy integration with normal code.
Lately, I’ve shifted more towards transforming ChatGPT’s outputs. By orchestrating multiple prompts and adding human influence, I can obtain responses that ChatGPT alone likely wouldn’t have come up with. Though, this has to be balanced with giving it the freedom to pursue a different thought process.
What method did you use to generate only JSON? I’m using it (gpt3.5-turbo) in a prototype application, and even with giving it an example (one-shot prompting) and telling it to only output JSON, it sometimes gives me invalid results. I’ve read that the new function-calling feature is still not guaranteed to produce valid json. Microsoft’s “guidance” (https://github.com/microsoft/guidance) looks like what I need, but I haven’t got around to trying it yet.