Opinion

We put our web data onto ChatGPT. Here’s what we found.

We took our website and fed it into ChatGPT to find out what there was to find out. Here it is.  Ask anything you like. 

Neato

Everyone already knows ChatGPT is cool. It’s a technological knockout. But is it endgame AI? No. Unequivocally no. It’s a piece of “Narrow Artificial Intelligence” that is a pretty decent step toward “General Artificial Intelligence”.

Narrow AI is like facial recognition. It can recognize a face, probably better than you. Going outside the boundaries of what it does, is just hyperbole. So anytime someone starts talking about how ChatGPT has a conscience and is trapped, or is a silver bullet for some business thing, use caution. It’s just not all that. At least not yet.

How do I know? Here’s what we did.

We scraped our website. Pumped it into ChatGPT. It’s kind of like jamming our website into ChatGPT’s short-term memory. That’s how it answers your questions. Want to know if K3 connects to Snowflake? That’s coming from short-term memory. Want to know how to bake sourdough bread? Our website will answer that too but from its long-term memory.

Here’s what we found out.

It can write well (and not well). It can write code It can summarize better than a fresh college grad. These are the narrow things it absolutely excels at. But here is something we did not expect. It can make a guess. I can’t believe it either. ChatGPT is a good guesser.


We asked ChatGPT, does K3 support webhooks? What is a webhook? Don’t worry about that.  What matters is what it said. “Yes, K3 supports webhooks”. Ok, what is the big deal? K3 does in fact support webhooks. The big deal is that we never told it that K3 supports webhooks. It guessed. 

That right there is the leap.

Being a good writer is a darn good trick. But the real leap here is that ChatGPT knows how to hold the context of an idea. We fed it enough information about K3 that it was able to contextualize the software. It has an idea in its head about what K3 does. Does a thing like K3 have webhooks? Yes. (ChatGPT calls this the temperature setting.

Thinking of using ChatGPT. Here is what you need to know:

ChatGPT’s Short-Term Memory:

The short-term memory is not so great. Remember, short-term memory is where you put your context. Maybe 3000 words. This will get better over time and there will undoubtedly be licenses to use it on a business scale, but it’s nowhere near that yet. Oh, and there is another challenge. That short-term memory? It’s really really short. On every request, you have to send the entire body of data you want ChatGPT to hold in short-term memory. Yes, for every question you ask on our website, we send your question to the API preceded with ALL the information about K3.  This is just not scaleable.

Risky Data Treatment:

DANGER! ChatGPT is NOT, I repeat NOT ready for any type of non-public data. The security is not there. (e.g.SSO).  You are handing over data to be used as they like. The takeaway here is that ChatGPT needs to be treated as an unsecured toy. Under no circumstances should anyone put any company data into it without the green light from your company information security officer (CISO).

It's Easy:

We probably spent two hours setting this up. It’s not hard at all.

Bursting With Potential:

Why’d Microsoft dive in with 10 billion? Sure, Bing becomes a better Google right out of the gate. But, MSFT is an enterprise company. Think of all the petabytes of Sharepoint data sitting out there. You know who goes looking for information in Sharepoint? Someone who is forced at gunpoint. With ChatGPT’s current functionality, all of that data can be easily accessed. The takeaway here is to watch it closely.

Better ‘Fu’ But At A Price:

You know that person in the office that is hell on wheels with google? Before you can get it out of your mouth they have the answer. That is a trait techno-nerds call “Google Fu”. ChatGPT makes high levels of Fu much more accessible. But there is a cost. No literally, there is a cost. The way that ChatGPT is set up, you are going to pay for it. Not much but if you want to use it you will pay. I promise you that. 

The Coding:

It’s a solid coder. So we asked, “Can you sling us some python to crawl our website?” It did. Can you add this thing to the code? It did. Can you write me a curl for the API? It did. The takeaway here is that it can take a level 2 coder to a level 5 coder in an instant. Can it take a level 5 coder to an 8? Probably not yet. A lot of low-level coding today is done by “Google Fu.” Googling to find the code you need. Again, ChatGPT has better Fu.

Flying Monkeys:

Last but not least, breakthroughs like this always release a hoard of flying monkeys. Everyone feels now compelled to slap AI onto their name. Talk about how they have solved the most intractable problems with ChatGPT. Might happen. But based on where ChatGPT is right now? Sorry, no. At least not right now.  But watch closely. This space is moving fast. But for now stick to Missouri. It’s the “Show Me” state. Accept neither wooden nickels or powerpoint slides.