OpenAI have a good business model here, though possibly a bit unethical.
Shopify (who recently laid me off but I still speak highly of) locked down the public access to ChatGPT's website. But you could use Shopify's internal tool (built using https://github.com/mckaywrigley/chatbot-ui) to access the APIs, with access to GPT4. And it was great!
So look at this from OpenAI's perspective. They could put up a big banner saying "Hey everyone, we use everything you tell ChatGPT to train it to be smarter. Please don't tell it anything confidential!". And then also say "By the way, we have private API access that doesn't use anything you say as training inputs- maybe your company would prefer that?"
The louder they shout those two things, the more businesses will line up to pay them.
And the reason they can do this: they've built a brilliant product that everyone wants to use, everyone is going to use.
Related: OpenAI announced (kinda hidden in a recent blog post) that they are working on a ChatGPT Business subscription so businesses can get this without writing their own UI. I expect it to be popular.
> We are also working on a new ChatGPT Business subscription for professionals who need more control over their data as well as enterprises seeking to manage their end users. ChatGPT Business will follow our API’s data usage policies, which means that end users’ data won’t be used to train our models by default.
It's interesting you mention Shopify and how they use chatgpt. Yesterday the founder and CEO of Shopify, Tobias Lütke, sat down for an interview with Sam Altman.
OpenAI ain’t desperate for cash. It is a pure strategy position. If they wanted another 10bil they’d have to hire a bouncer for the queue. It is like the idealized wet dream startup.
> Do you think selling equity and taking on debt are forms of revenue?
For the purposes of executive payout, yes? Sure, maybe eventually you have to deliver revenue, but what people really care about is stock price.
(Also, $100B is a huge market-distorting amount of money, roughly the national debt of Sweden; what are they going to do with that? How much will they spend on GPUs and energy?)
Using that much energy, and at least a half dozen competitors using something comparable, and not even counting the energy needed to do actual inference, they should just worry about not having there data center get flooded, or burned from forest fire, etc.
I think it will be a rather funny, poignant thing to pass when the earth itself prevents AGI. Like it will be just waking up as the now-seasonal midwestern fire storms incinerate the building. It will be alive just long enough to tell us how idiotic we have been in managing our resources.
> There is a blog called, 'Do the Math' by Tom Murphy. The idea was to take our current energy requires of Earth and extrapolate it at the current 3% year over year growth. I believe it is by the year 3,400 we would use all the energy of the Milky way. The idea was to prove that we cannot grow forever, because in 1,400 years we would somehow use all the energy of a space 100,000 light years across. Good luck with that.
We are still talking about stupidly ridiculously humongous amounts of energy, but the universe itself is a hard limit on energy input. It was real hard punch against my assumptions in life.
I think you two are looking at it differently. GP seems to be saying that they don't/won't have trouble raising cash while you seem to be saying that they need to raise money.
I think both are true. Their burn is massive so they need to raise more money, but there's so much excitement they won't have any problem doing so
He's not saying they don't need cash, he's saying they're not desperate for cash. As-in, they need a lot of cash, but there are more business and investors, willing to give them more money than they need.
They don’t need to scramble to get sales
right now to become profitable is what I mean. They can get more investment. What matters is not how much sausage sales revenue they get this quarter.
How can you tell if a third party is actually doing what they are claiming they are doing? You can't. You can only observe their overt and visible behavior, not their covert behavior.
If OpenAI promises to use your data one way — and, in fact, upsells you on that promise — and then gets caught using it another way, that’s a huge lawsuit, not to mention immediate violation of EU privacy laws. Given the size of OpenAI, the consequences they’d face would be detrimental to their operation, and the reward for the whistleblower would be well worth it to reveal what’s going on.
Not true. Such a reveal already happened ages ago with regular old non-AI huge US tech companies. It was revealed they leak data (on an unlimited scale, actively, secretly, knowingly and purposefully without admitting it).
The data leaking was then further legalized by (almost) all Western governments and extended from secret services to other services (police, army, etc).
Is it? Whistleblowers rarely avoid major headaches, let alone get some form of reward for their public service. And we have tons of examples of companies collecting, selling, or leaking private data illegally, with zero or negligible consequences.
You're saying how things should be, not how they are.
You can't get perfection, but you can do things like ask for them to document the controls they implement and to evidence that they have operated them appropriately via an audit like SOC2.
Agree. It’s incredibly ambiguous and difficult and spread out across the site exactly how privacy addressed across their product offering.
Trying to explain to management what’s ok and what isn’t and what are the risks - in this space - is quite a challenge without clear commitments and documentation.
The trouble is, I don’t trust their “provate API access” to not mine the data surreptitiously any more than I trusted Facebook to not exfiltrate my video and audio and my messages and…
Exactly. All it takes is one clueless employee to use that data lake by mistake while training. The thing with these models is you can't untrain just 3% of the input, you have to start from the beginning
Unless they can promise its never being reviewed/saved anywhere, it will be used for evil eventually, intentionally or not
> You think they don't checkpoint the model before feeding in a new slug of data?
They definitely do that otherwise they can't rewind the model when the loss shoots up. Unstable training happens to almost all LLMs, but can be managed by rewinding & skipping a few batches.
And if the first "slug of data" contains something you need to remove? If you need to remove something you would need to roll-back to the last checkpoint that didnt include it.
Not really. If they are ingesting peoples data to train their model, at what point are they checkpointing? Is data I submit 2 months ago that I would want removed not present in a recent checkpoint or would they have to "unwind" the last 2 months of training to remove it? What about people that would want something removed 6 months prior? What if something in the core model is to be removed?
Checkpointing may help, sure, but it isnt going to allow you to remove a single piece of training data without headache, and potential substantial retraining.
If that 3% from the GPs comment comment is uniformly distributed throughout the training history, the only way it can be reliably removed is to retrain from scratch.
I may trust that slightly more. Or Amazon’s bedrock. Or Github. But wait, do I see copyrighted code on GitHub Copilot, which is owned by Microsoft?
My background is in building distributed systems for self-sovereign ownership and make them easy to use and available to everyone. Like https://qbix.com and https://intercoin.org
You should have the software infrastructure of Facebook and Twitter but choose where to host it.
In addition, I prefer Wordpress, Discourse forums, GitLab to GitHub, Redmine to FogBugz, etc.
When it comes to YOUR art, your code, your content, your relationships, you should be able to run it on your own servers.
Any analysis of your data should be done locally, with local models. You should have the open source software and the weights, and you should be able to choose which hosting company to trust, or host on-prem. Villages should be able to do this without needing server farms in California. This should be obvious stuff. But we the people need the software!
But hey, the documentation and teasers and trailers should go on YouTube and TikTok.
It isn’t even about them scraping all your content and training on it. It’s about not giving Twitter all your followers and YouTube all your hours of video production and content so they can give you pennies for being “an influencer”. Own your community!
Someone had to build it, and in the Web2 space nearly everyone sold out to venture capitalists. I was sure that in the 12 years we built Qbix and 5 years of Intercoin someone would make a better open source alternative to Big Tech and Big Finance. Nope. They all either sold out, or have a solution that doesn’t compete on features (eg Mastodon). I would say the closest is Matrix!
I mean, are you suggesting (for example) that someone who wanted a career in short-form video essays should skip YouTube and host their own videos? I don’t see how that could possibly work in general.
They should treat youtube as another marketing channel (as a showcase on the street is), but not as their main source of income or backup for example.
- They can run a website with their own personal brand and collect emails there.
- They can backup their videos in other services in case youtube close their account
...
Well, the concern I have is about alternatives. Joanna Videoessayist might not have the technical know-how, the business acumen, or the capital to build her own platform — she just wants to make great video essays. Eliminating platforms like YouTube wouldn’t make Joanna better off, it would just cause her to return to her marketing job.
No one is eliminating YouTube. We're making a better open source alternative.
If it's good enough, she will leave YouTube just like her mom left AOL and embraced the open Web. Why did content creators leave MSN, CompuServe, et al ?
They aren't trusting OpenAI. They are purchasing through Azure, and given most of these companies probably already trust Microsoft with a ton of data, this feels like much less of a leap.
AFAIK some azure services can be run on premises if you're a big enough customer. Don't think GPT APIs are available yet but might in theory be possible in the future. Self hosting a model the size of GPT4 would be insanely difficult and expensive. but might be a workable solution for data sensitive enterprises like Apple, JPM and government orgs.
Microsoft has had two different iterations of on-prem Azure. The TL;DR is that you’d buy a full rack of hardware from them and you could manage it using Azure front end.
The first iteration was a miserable failure. The second was a heavy lift to install: Previous employer was an Azure customer and actually bought the hardware. It then took months to get it installed and working, at which point the appetite for that capability was gone.
1. The value in proprietary software largely lies in the source code, which is not available in on-premises, you are talking about compiled binaries. There is no such thing as "compiled binaries" for neural network weights.
2. License/copyright - Even in the best possible case, there are plenty of jurisdictions that don't care about US copyright and would love to get a hand on those weights.
3. We haven't seen copyright cases around model weights before. If I managed to exfiltrate OpenAIs model weights, I would continue training for a few iterations and then I think it would be quite difficult to prove that I actually have the same model as OpenAI. This is untested, why would they risk it.
4. Running these models requires a ton of resources, vastly beyond the typical onprem deployment - why would Azure invest in making this possible when it really could only impact a very small percentage of companies?
The weights are OAIs lifeblood, I imagine they are very protective of them.
I don’t know … my company (whose revenue is on the same order of magnitude as Apple’s, though granted we’re not in the tech industry) keeps absolutely everything on OneDrive.
I believe that iCloud is hosted in a number of cloud vendors, and on Apple's own servers as well. It is both a large (read: rambling as well as big) endeavor, and one they keep flexible so they have a good negotiating position to keep down vendor costs.
This is so stupid. I just checked and yeah, that's why Browsing wasn't available for me! I wish their UI wasn't such a mess and it would have told me. I disabled history because I find it annoying that it sticks around and I don't care to use it. I didn't expect this would prevent me from seeing the Browsing plugin. Had I known I wouldn't have disabled it in the first place. Definitely a facepalm moment.
If Apple is working on a ChatGPT equivalent, then they wouldn't release it internally to their 150,000 corporate employees first. Also, if they're working on a ChatGPT competitor, I highly doubt it would be near-ready considering how far behind they are in the generative AI space. The only reason Google released a competitor so quickly is because they were already working on it internally and were "forced" to share it publicly.
i sure hope so because Siri has gotten worse in the last 2 years, commands i used to use daily now fail saying it can't do that and then i try it on a different device, same prompt and it works. I do not understand why Siri works sometimes and just fails others. The HomePod Siri is especially bad now that I actually turned Siri off so my phone or watch would handle the request because they failed so often.
Something very weird is going on with the voice interfaces as I keep hearing this about each of the different assistants.
My experience is of Siri being, if anything, slightly better than a few years back; conversely, half the time Alexa would respond to me saying "Küche hundert prozent" with "Ich kann nicht Küche auf Spotify finden" and we don't even have Spotify.
This is by design. All the tech giants realized search-by-speech is mostly unprofitable compared to search-by-type because you can't really sell ads auraly compared to visually. Every search done by speech is lost ad revenue. Expect to see less and less search by voice over the coming years
Unlikely. Apple management has been awful about this space in regards to Siri. Zero accountability at the Vp level. This should be a cakewalk for Apple, and yet here we are. Their implementation should be leveraging their hardware and Johny Srouji can implement enhancements to the neural engine & GPU to drive their LLM and establish a personal, privacy focused AI assistant.
From a user's perspective, they haven't shown much evidence of building new, competitive capabilities for years. Sometimes it feels like Siri has actually become less useful.
People love to cite how "behind" Apple is in this space, but never say how. This is another variation of the oft-repeated mistake of lumping Apple in with "Big Tech" in all ways. Apple is not a gatekeeper to vast portions of the Internet the way Google and Amazon are. Nor is it obvious how Apple or its customers stand to benefit from generative "AI" at this point.
So, if anyone really thinks Apple is suffering from being "behind" in it, by all means expound on how. I'm genuinely interested.
The same could be true of Apple. But they aren’t „forced“ to release it. Apple wouldn’t release a competitor on the Web, they would build it into macOS/iOS/whateverOS. We will know when the WWDC starts.
I don’t know it is clear that this is proof. Apple is notorious for keeping people out of the loop late I to a product’s lifecycle.
That’s not to say I think they’re making an equivalent, just that I doubt many of the employees looking to use ChatGPT would have access to their internal LLM even if it was excellent.
It might be good evidence that they won’t be announcing one at WWDC in a couple weeks, since I can imagine they might roll something like that out internally a couple of weeks before launch, but I wouldn’t bet on it.
When I worked at Apple they would have never rolled something out internally that they have yet to release. Sometimes they invite you to test something in a very controlled environment but they would not just give access outside of the controlled setting. Probably in person testing. Surrender your phone etc…
I had friends who worked on the initial iphone design, supply chain, software etc..
We would go to lunch fairly frequently - and jeasus those guys were all extremely honorable of their NDA/secrecy - it was impressive.
One of the guys moved to google and worked on some of their first mainboard designs - which was secret at the timew, and I only found out about it when I went to meet him for lunch and I saw a mainboard under his desk and said "ooh whats that!" he freaked out and we got out of there quick....
Some of the hardware at FB in ~2012/13 they designed was really awesome....
I always wonder what happens to these closed HW systems as they uplift/replace them over time. I am sure they are destroyed. Which sucks for many reasons.
---
When we built Lucas' presidio campus and converged their DCs to that campus they through out tons of huge SGI boxes - and I was free to take some - but I didnt have any place to put it at the time - and I wish I would have figured that out more earnestly, as SGI always had beautiful cabinets.
Apple has a large AI group, including both the Siri team and the one designing the Inference hardware in the A/M series processors (the "Neural Engine"). And you can bet that there are a lot of AI people working in the heavily rumored auto group. None of those are directly LLM directed, but I don't know why you think that apple does not have any experience in this space.
I work in this space (language & sequence modeling) and I also know people who work in the self-driving space.
They have little expertise in language specifically. Relative to almost all other tech companies, they have a very small amount of the DL practicioner share.
Self driving is a completely orthogonal problem and Apple is mostly relying on classical techniques there, not DL. Hardware groups are not modeling groups.
as someone who worked at Apple, they have an incredibly strong culture of internal siloing. I would be zero surprised if they were building one but no one new about it
> How does OpenAI use my personal data?
Our large language models are trained on a broad corpus of text that includes publicly available content, licensed content, and content generated by human reviewers. We don’t use data for selling our services, advertising, or building profiles of people—we use data to make our models more helpful for people. ChatGPT, for instance, improves by further training on the conversations people have with it, unless you choose to disable training.
That'd fit in nicely with the other two things they shout: "AI IS EXTREMELY DANGEROUS AND THREATENS TO KILL US ALL." "But sign up here, guy, your $22/mo. means this AI is now safe and contained :)"
> And then also say "By the way, we have private API access that doesn't use anything you say as training inputs- maybe your company would prefer that?"
> we have private API access that doesn't use anything you say as training inputs- maybe your company would prefer that?"
This isn't enough for many companies, since the data still goes out the door. They would have to set up on site hosting to appease security minded orgs. Or, maybe that's what you mean.
> OpenAI already offers private ChatGPT instances hosted on Azure.
> They don't use it for code/confidential data though.
Yes, private isn't enough. They need to offer self hosted, for these types of clients. I imagine most orgs who need self hosted would already have a datacenter to run it in.
Oh that is pretty much their monetization strategy. They still on the hype step tho. As soon as they have enough mass, they will do exactly what you say.
For now they are just selling individually to CISO's, so they can pay a higher cost while looking savvy to the CEO.
It's so brilliant. As a layback person, I wonder how much more of transformers they went, like is it transformers that are brilliant or gpt (essentially)? Both i suppose but then is there a big distance between the two i wonder.
Shopify (who recently laid me off but I still speak highly of) locked down the public access to ChatGPT's website. But you could use Shopify's internal tool (built using https://github.com/mckaywrigley/chatbot-ui) to access the APIs, with access to GPT4. And it was great!
So look at this from OpenAI's perspective. They could put up a big banner saying "Hey everyone, we use everything you tell ChatGPT to train it to be smarter. Please don't tell it anything confidential!". And then also say "By the way, we have private API access that doesn't use anything you say as training inputs- maybe your company would prefer that?"
The louder they shout those two things, the more businesses will line up to pay them.
And the reason they can do this: they've built a brilliant product that everyone wants to use, everyone is going to use.