Are we close to generating an untextured room with a single untextured T-pose NPC?
This is unnecessary. The environment and characters can all still be built and rigged using modern hand modeling and set design techniques. The get here would be the animations. All animations, and I mean all, are just a bunch of splines and interpolations in the engine. Curve fitting is something that AI’s have been getting trained to do for 50 years. It is a solved problem. What is not solved, and the reason we don’t see a ton of AI animation tools already is a lack of sufficiently large sample sets of good animations attached to a variety of rigs and a method of training the AI to be environmentally aware. I imagine the latter could be solved using computer vision techniques with virtual sensors like what is used for Crowds. It is actually a hypothetical problem that I am planning on tackling after my current project.
Or would it be using a language model to obfuscate using an RNG?
This is almost insulting to the entire field of AI development, not to mention the mathematical fields of Probability Theory and Statistics. While AI models do function on Game Theory and Probability Theory models and are probabilistic, they are most definitely not any form of RNG. If you have that mindset, I suggest you do some more reading on the topics in scholarly publications or textbooks. Skip the pop-sci articles and blog posts.
As to your questions on timing, it may be this year, it may be 20 years. 2 years ago the idea of making photoreal images with AI was a pipe dream, then along comes Stable Diffusion and in less than 18 months we have gone from making passable images of a cat to an almost fully art-directable toolset capable of creating coherent videos. This is an amazing progressive leap and AIs in general have become more diverse because of it. It is a testament to just how fast something like this can grow given the right FOSS architecture and public interest. My guess is that it is closer to the 2 to 3-year range for a coherent world-building AI. That is not modeling, rigging, or animating, just the textual stuff. Story, relationships, lore, history, etc. My first tool I am trying to build is a world-building assistant for TTRPG GMs if that gives you a clearer picture of what I’m talking about.
Finally, the geometry generation times. You mention your experience, but I am struggling to pin down exactly what that entails for you. I mentioned using procedural modeling and having an AI that decides the parameters of the procedural when doing the modeling. In this setup the modeling is all mathematics and is done instantly. It can even include procedurally defined animations and affects that are able to be generated on the fly in microseconds. If you have 3D experience, I would suggest checking out Houdini ( https://www.sidefx.com ) to get a better grip on what is possible with proceduralism. They also have tools for doing rigging and animation that an AI could directly interface with, which can be utilized in game development realms as well as VFX.
You misread me when i said RNG - I was saying using a language model to appear as if it’s generating a quest on the fly, but actually its just picking Kill [5-30] [Goblins / Demons / Beasts / Mechs] from a list is not “AI generated quests”
And you misunderstood what I was saying about being a Dev - I said I was not a dev.
I know the following style of reply is a bit rude - but I want to highlight how I said that much of what tech companies are promising with AI is an empty promise that will only lead to semantic satiation of the term “AI” leading it to become a term of derision
can be built with hand modeling techniques
so not AI.
lack of sample sets
so AI can’t do it (yet)
[no] method of training the AI to be env aware
so AI can’t do it (possibly ever)
it may be 20 years
so unlikely to be a reliable marketing promise to consumers in this generation or the next
my guess is it’s closer to 3 years
sounds like marketing bumpf
…just the textual stuff
so not AI generated quests in game as originally promised…
Look, I really don’t mean to pick a fight with you - but saying “I’m working on AI generated quests!” to mean “I’m working on using an existing LLM to create text-based lore entries on my world” is a very different expectation to reality ratio.
Your response style is not rude. It is disingenuous confirmation hunting cherry picking. I addressed everything in your response in detail, but you latched on to the qualifiers and clarifications because those are the parts that satisfy your confirmation bias, even though in the full context they mean quite the opposite. I also am recognizing that you are not a dev, not a game designer, not an AI architect nor mathematician, or a computer scientist. All of this means that the scope and breadth of your understanding of the topic you are attempting to belittle and demean is myopic at best. It is obvious that you do not understand what a “quest” in a game is nor what it takes to craft or write one. It is clear that you have 0 understanding of how LLM or AIs as a larger topic function or generate information. You deign to belittle the work that I am doing on this topic without asking a single question or clarifying a single detail from me. I’m sure you are getting a nosebleed for your perch on Mt. Dunning-Kruger. I can see you from my position on the adjacent slope. I will humor you though, since you seem to at least care about the topic, even though you seem utterly incapable of recognizing when something should be informative and educational.
What is a “quest” in a game:
The abstract of this article covers some key concepts of both current anf potential future paradigms of quest design in video games. Currently, quests, even handcrafted ones, consist of a list of tasks. Lists of tasks are text. Well-crafted ones include lore hooks, personal interactions with a variety of NPCs, tasks that are meaningful to the world, and rewards/results that either expand the player’s capabilities or draw the player into the world and the story. Components needed to craft a quest are a means of initiating the quest for the player (quest givers, discovered information like a letter or journal, an event that happens in the player’s presence), dialog, goals/tasks, and a location. For a well-made persistent world, locations, and NPCs are persistent as well. They don’t need to be AI-crafted on the fly. They should have been created when the game was made, or at least some period long before the players interacted with them. What can be crafted as a just-in-time or an emergent experience is the interaction with the player. Dialog with lore hooks that give the quest grounded footing in the world. Interactions with other NPCs which are based with the relationships of the NPCs with each other instead of just the player, etc. All of this is textual in nature. Dialog, relationships, interaction. The secondary stuff, like animations, could be handled in a lot of clever ways. They could be hand-crafted by animators and an AI could be trained on what animation links to what interaction type. The AI then selects and blends the canned animations when it is interacting with the player. And before you have something to say about that, it is literally how animation is done now by people. I absolutely have authority on this topic as I have a graduate degree in animation and visual effects.
The AI I am working to create currently is being designed around the goal of making a consistent world with complex relationships and personalities so quests can be written that are self-consistent and coherent with the world. It is not worried about writing dialog or generating rewards, just making a world and quests that are cogent and have minimal plot holes.
What isn’t part of a quest?
Geometry, models, textures, particles, VFX, rooms, people, trees, animals… basically anything that has physical form or defines a physical characteristic of an object. These things may be involved in the completion of the tasks themselves, but they are all separate from the quest. As an analog, your wife hands you a grocery list and asks you to go to the store to pick up the things she needs for dinner. This would constitute your bog-standard fetch quest in a video game. The quest is to obtain the listed items. All of the items exist independently of the quest. The grocer and cashier at the till do as well. As does your wife. The only thing that the quest consists of is the request which is made, and the list which is requested. Everything else does not need to be created, it already exists. I hope that clears up your misunderstanding of the topic.
You can erase the entire concept of modeling or texturing from your mind in relation to an AI creating quests. I mentioned it because I recognise that some objects may need to be created. Perhaps your wife asks for a product which the store did not contain already. So in a gaming context, this should be cleaned up so the quest is able to be completed. The product needs to be created. You can place limits on the list of items so they are all canned goods. Now you have a rudimentary prototype object that can be anything. Canned beans, sure. Canned orc tongue, why not? All that is needed is a label and a spot on a shelf. SD can create a label in a few seconds while the player moves from his home to the store and an observation-aware procedural system can stock it on the shelves when no players are looking and inform the AI driving the cashier of the existence and location of the item. Nothing needed modeled. Hell, all that was really needed was a procedural text generator to make a label for the can, no AI was needed for that. What the AI is needed for to make the trip to the grocery feel right is the request from your wife, the conversation with the cashier at checkout, and the smile and thanks from your wife when you got home with the items.
If more modeling is needed, procedural modeling takes care of that. Artists create prototype objects with fixed bounds on parameters and an AI is trained on how to set those parameters, so whenever a quest needs an item, the quest generation AI requests the type if item and the context in which it will be used, and the modeling AI interprets the request, prepares the correct prototype, determines the contextually appropriate parameters for it, then places it where the quest giver needs it. No muss, no fuss. Eventually, there will be SD analogs for 3D geometry and generative modeling. It is a field of active research at IBM, nVidia, AMD, Meta, and many others. It will happen, and it will happen sooner than anyone will be ready for it, much like SD. But for now, it is overkill. AIs can use procedural modeling to adequately furnish and populate a world without much overhead. I think I may play with that as well. Rip the character creator from the likes of Skyrim or Starfield and train an AI on faces and the character creator parameterization and have it go wild. The AI is still creating the characters, it is just using the same tool as the player to do so.
You took all that to say what I was originally saying 4 or 5 comments higher in the chain: for idle chitchat yes AI could probably do it, but it can’t make quests on the level of CP77’s crucifixion quest, all it can do is choose from preselected parameters - but we don’t need to do that. We have radiant quests already (as mentioned, Preston Garvey…)
Also your character creator randomizer doesn’t need AI, it just needs a restrictive algorithm (maybe not even) not to get too random - ie don’t put lizard hands on a human body, limit how big you can make the nose, ensure skin tone is even…)
Same for your canned goods creator, it doesn’t need AI it just needs a pick list and an RNG. You don’t even need AI to make the picklist, you can just scrape a list of most popular canned goods from Wikipedia or some stats site, and a list of your in-game races/affiliations/species
When you say that in-game characters, geography and props aren’t needed, you’re wrong — going to New locations, meeting new people and seeing unique things is part of what makes a quest interesting. You must of heard of gamers complaining about reused assets and reskinned characters and guns as rewards- adding a system that puts more “kill 5 goblins to get the same sword you’ve got but slightly more blue” will only make the game worse not better.
So, no, AI can’t “do quests” as most people understand them, it can only create busy work, which is considered one of the things dragging the gaming industry down right now.
Ok, I’m done trying to educate you. You, by your own admission, are lay on literally every topic that you are speaking about and I am literally an expert. I have put in my thousand hours on modeling, animation, and VFX, as well as hundreds of hours on quest design for TTRPGs. I have a master’s degree in Animation and a bachelor’s degree in Applied Mathematics. I am professionally a developer and am actively designing AIs. Why the hell are you arguing with me on what is and is not possible? You know nothing but what you have read in a few pop-sci articles and blogs along with some observations from playing games. I have all of that too, plus everything else. Your opinions are based on incorrect assumptions that invalidate them from the start. If you can provide me a single scholarly article, published in a journal in the last 2 years that supports a single one of your assertions, I will yield on whatever you are able to support. Hell, I will even take a passage from a textbook on AI design published in the last year. I have already provided support for some of my assertions. Please provide some of yours. Again, scholarly publication. Not blog. Not pop-sci articles. A recent whitepaper from a research group is also acceptable.
I dont have to understand the deep complexities of those things because we haven’t even got past the basics, and where the key issue lies.
I dont need a degree in animation to confidently tell you that AI isn’t in the place where it can generate animations at scale for a playable experience without significant hand-holding. Is that true or not? In your reply above you agree with me saying AI can only pick from a list of already-created animations, which makes it no different to an RNG.
There aren’t any scholarly articles about “AI can’t make video games for you” because scholarly articles are about proving things can happen. Can you provide scholarly articles on consumer-ready video games that have been created with majority AI parts? After all extraordinary claims require extraordinary proof.
If you can do this- why haven’t you? (“The technology isn’t there yet!” I know that’s been my argument from the start)
This is unnecessary. The environment and characters can all still be built and rigged using modern hand modeling and set design techniques. The get here would be the animations. All animations, and I mean all, are just a bunch of splines and interpolations in the engine. Curve fitting is something that AI’s have been getting trained to do for 50 years. It is a solved problem. What is not solved, and the reason we don’t see a ton of AI animation tools already is a lack of sufficiently large sample sets of good animations attached to a variety of rigs and a method of training the AI to be environmentally aware. I imagine the latter could be solved using computer vision techniques with virtual sensors like what is used for Crowds. It is actually a hypothetical problem that I am planning on tackling after my current project.
This is almost insulting to the entire field of AI development, not to mention the mathematical fields of Probability Theory and Statistics. While AI models do function on Game Theory and Probability Theory models and are probabilistic, they are most definitely not any form of RNG. If you have that mindset, I suggest you do some more reading on the topics in scholarly publications or textbooks. Skip the pop-sci articles and blog posts.
As to your questions on timing, it may be this year, it may be 20 years. 2 years ago the idea of making photoreal images with AI was a pipe dream, then along comes Stable Diffusion and in less than 18 months we have gone from making passable images of a cat to an almost fully art-directable toolset capable of creating coherent videos. This is an amazing progressive leap and AIs in general have become more diverse because of it. It is a testament to just how fast something like this can grow given the right FOSS architecture and public interest. My guess is that it is closer to the 2 to 3-year range for a coherent world-building AI. That is not modeling, rigging, or animating, just the textual stuff. Story, relationships, lore, history, etc. My first tool I am trying to build is a world-building assistant for TTRPG GMs if that gives you a clearer picture of what I’m talking about.
Finally, the geometry generation times. You mention your experience, but I am struggling to pin down exactly what that entails for you. I mentioned using procedural modeling and having an AI that decides the parameters of the procedural when doing the modeling. In this setup the modeling is all mathematics and is done instantly. It can even include procedurally defined animations and affects that are able to be generated on the fly in microseconds. If you have 3D experience, I would suggest checking out Houdini ( https://www.sidefx.com ) to get a better grip on what is possible with proceduralism. They also have tools for doing rigging and animation that an AI could directly interface with, which can be utilized in game development realms as well as VFX.
You misread me when i said RNG - I was saying using a language model to appear as if it’s generating a quest on the fly, but actually its just picking Kill [5-30] [Goblins / Demons / Beasts / Mechs] from a list is not “AI generated quests”
And you misunderstood what I was saying about being a Dev - I said I was not a dev.
I know the following style of reply is a bit rude - but I want to highlight how I said that much of what tech companies are promising with AI is an empty promise that will only lead to semantic satiation of the term “AI” leading it to become a term of derision
so not AI.
so AI can’t do it (yet)
so AI can’t do it (possibly ever)
so unlikely to be a reliable marketing promise to consumers in this generation or the next
sounds like marketing bumpf
so not AI generated quests in game as originally promised…
Look, I really don’t mean to pick a fight with you - but saying “I’m working on AI generated quests!” to mean “I’m working on using an existing LLM to create text-based lore entries on my world” is a very different expectation to reality ratio.
Your response style is not rude. It is disingenuous confirmation hunting cherry picking. I addressed everything in your response in detail, but you latched on to the qualifiers and clarifications because those are the parts that satisfy your confirmation bias, even though in the full context they mean quite the opposite. I also am recognizing that you are not a dev, not a game designer, not an AI architect nor mathematician, or a computer scientist. All of this means that the scope and breadth of your understanding of the topic you are attempting to belittle and demean is myopic at best. It is obvious that you do not understand what a “quest” in a game is nor what it takes to craft or write one. It is clear that you have 0 understanding of how LLM or AIs as a larger topic function or generate information. You deign to belittle the work that I am doing on this topic without asking a single question or clarifying a single detail from me. I’m sure you are getting a nosebleed for your perch on Mt. Dunning-Kruger. I can see you from my position on the adjacent slope. I will humor you though, since you seem to at least care about the topic, even though you seem utterly incapable of recognizing when something should be informative and educational.
What is a “quest” in a game: The abstract of this article covers some key concepts of both current anf potential future paradigms of quest design in video games. Currently, quests, even handcrafted ones, consist of a list of tasks. Lists of tasks are text. Well-crafted ones include lore hooks, personal interactions with a variety of NPCs, tasks that are meaningful to the world, and rewards/results that either expand the player’s capabilities or draw the player into the world and the story. Components needed to craft a quest are a means of initiating the quest for the player (quest givers, discovered information like a letter or journal, an event that happens in the player’s presence), dialog, goals/tasks, and a location. For a well-made persistent world, locations, and NPCs are persistent as well. They don’t need to be AI-crafted on the fly. They should have been created when the game was made, or at least some period long before the players interacted with them. What can be crafted as a just-in-time or an emergent experience is the interaction with the player. Dialog with lore hooks that give the quest grounded footing in the world. Interactions with other NPCs which are based with the relationships of the NPCs with each other instead of just the player, etc. All of this is textual in nature. Dialog, relationships, interaction. The secondary stuff, like animations, could be handled in a lot of clever ways. They could be hand-crafted by animators and an AI could be trained on what animation links to what interaction type. The AI then selects and blends the canned animations when it is interacting with the player. And before you have something to say about that, it is literally how animation is done now by people. I absolutely have authority on this topic as I have a graduate degree in animation and visual effects. The AI I am working to create currently is being designed around the goal of making a consistent world with complex relationships and personalities so quests can be written that are self-consistent and coherent with the world. It is not worried about writing dialog or generating rewards, just making a world and quests that are cogent and have minimal plot holes.
What isn’t part of a quest? Geometry, models, textures, particles, VFX, rooms, people, trees, animals… basically anything that has physical form or defines a physical characteristic of an object. These things may be involved in the completion of the tasks themselves, but they are all separate from the quest. As an analog, your wife hands you a grocery list and asks you to go to the store to pick up the things she needs for dinner. This would constitute your bog-standard fetch quest in a video game. The quest is to obtain the listed items. All of the items exist independently of the quest. The grocer and cashier at the till do as well. As does your wife. The only thing that the quest consists of is the request which is made, and the list which is requested. Everything else does not need to be created, it already exists. I hope that clears up your misunderstanding of the topic.
You can erase the entire concept of modeling or texturing from your mind in relation to an AI creating quests. I mentioned it because I recognise that some objects may need to be created. Perhaps your wife asks for a product which the store did not contain already. So in a gaming context, this should be cleaned up so the quest is able to be completed. The product needs to be created. You can place limits on the list of items so they are all canned goods. Now you have a rudimentary prototype object that can be anything. Canned beans, sure. Canned orc tongue, why not? All that is needed is a label and a spot on a shelf. SD can create a label in a few seconds while the player moves from his home to the store and an observation-aware procedural system can stock it on the shelves when no players are looking and inform the AI driving the cashier of the existence and location of the item. Nothing needed modeled. Hell, all that was really needed was a procedural text generator to make a label for the can, no AI was needed for that. What the AI is needed for to make the trip to the grocery feel right is the request from your wife, the conversation with the cashier at checkout, and the smile and thanks from your wife when you got home with the items.
If more modeling is needed, procedural modeling takes care of that. Artists create prototype objects with fixed bounds on parameters and an AI is trained on how to set those parameters, so whenever a quest needs an item, the quest generation AI requests the type if item and the context in which it will be used, and the modeling AI interprets the request, prepares the correct prototype, determines the contextually appropriate parameters for it, then places it where the quest giver needs it. No muss, no fuss. Eventually, there will be SD analogs for 3D geometry and generative modeling. It is a field of active research at IBM, nVidia, AMD, Meta, and many others. It will happen, and it will happen sooner than anyone will be ready for it, much like SD. But for now, it is overkill. AIs can use procedural modeling to adequately furnish and populate a world without much overhead. I think I may play with that as well. Rip the character creator from the likes of Skyrim or Starfield and train an AI on faces and the character creator parameterization and have it go wild. The AI is still creating the characters, it is just using the same tool as the player to do so.
You took all that to say what I was originally saying 4 or 5 comments higher in the chain: for idle chitchat yes AI could probably do it, but it can’t make quests on the level of CP77’s crucifixion quest, all it can do is choose from preselected parameters - but we don’t need to do that. We have radiant quests already (as mentioned, Preston Garvey…)
Also your character creator randomizer doesn’t need AI, it just needs a restrictive algorithm (maybe not even) not to get too random - ie don’t put lizard hands on a human body, limit how big you can make the nose, ensure skin tone is even…)
Same for your canned goods creator, it doesn’t need AI it just needs a pick list and an RNG. You don’t even need AI to make the picklist, you can just scrape a list of most popular canned goods from Wikipedia or some stats site, and a list of your in-game races/affiliations/species
When you say that in-game characters, geography and props aren’t needed, you’re wrong — going to New locations, meeting new people and seeing unique things is part of what makes a quest interesting. You must of heard of gamers complaining about reused assets and reskinned characters and guns as rewards- adding a system that puts more “kill 5 goblins to get the same sword you’ve got but slightly more blue” will only make the game worse not better.
So, no, AI can’t “do quests” as most people understand them, it can only create busy work, which is considered one of the things dragging the gaming industry down right now.
Ok, I’m done trying to educate you. You, by your own admission, are lay on literally every topic that you are speaking about and I am literally an expert. I have put in my thousand hours on modeling, animation, and VFX, as well as hundreds of hours on quest design for TTRPGs. I have a master’s degree in Animation and a bachelor’s degree in Applied Mathematics. I am professionally a developer and am actively designing AIs. Why the hell are you arguing with me on what is and is not possible? You know nothing but what you have read in a few pop-sci articles and blogs along with some observations from playing games. I have all of that too, plus everything else. Your opinions are based on incorrect assumptions that invalidate them from the start. If you can provide me a single scholarly article, published in a journal in the last 2 years that supports a single one of your assertions, I will yield on whatever you are able to support. Hell, I will even take a passage from a textbook on AI design published in the last year. I have already provided support for some of my assertions. Please provide some of yours. Again, scholarly publication. Not blog. Not pop-sci articles. A recent whitepaper from a research group is also acceptable.
I dont have to understand the deep complexities of those things because we haven’t even got past the basics, and where the key issue lies.
I dont need a degree in animation to confidently tell you that AI isn’t in the place where it can generate animations at scale for a playable experience without significant hand-holding. Is that true or not? In your reply above you agree with me saying AI can only pick from a list of already-created animations, which makes it no different to an RNG.
There aren’t any scholarly articles about “AI can’t make video games for you” because scholarly articles are about proving things can happen. Can you provide scholarly articles on consumer-ready video games that have been created with majority AI parts? After all extraordinary claims require extraordinary proof.
If you can do this- why haven’t you? (“The technology isn’t there yet!” I know that’s been my argument from the start)