Twitxy Saved

cole murray

Verified account Ramp

@colemurray

principal eng @tryramp • building glass • prev: OpenInspect
San Francisco, CA murraycole.com Joined February 2015
920 Following 6.3K Followers 9.6K Posts

Latest posts 5

Pinned post

cole murray Verified account @colemurray
Advice given to someone asking about AI Consulting:

I don't think an ML background is required to be successful in AI consulting, but obviously helps. I think the biggest "skill" learned in ML is how to successfully do feedback loops in a system.

In an ML system, this typically involves cleaning data, making model tweaks, performance evals etc. In LLMs, in nearly every case you won't be fine-tuning the model, but iterating on prompts is a very similar workflow.

I do think it would be helpful to at least get a high-level learning of how the models "actually" work and become familiar with the basic terms. e.g. tokens, transformers, attention, what happens on each input -> output iteration as the model is predicting. You don't need to know the underlying math (helpful though), but having the understanding of what is happening is helpful.

Most of the AI consulting market is more on full-stack / product development skills and less ML. This isn't the most lucrative opportunities, but they are available in abundance.

Major areas now and over the next year:
- RAG: this is basically just glorified search lol. Useful in many contexts but severely overhyped

- Agents: The models aren't quite there yet IMO for this to be useful, but in 2025 I think this will be a major theme and a HUGE area of interest/investment. Becoming good at this will be valuable.

- Evals: Performance evaluations are a relatively untapped market. Most AI products you see today are flying by the seat of their pants. Without eval metrics, you can't truly know if your prompt changes are improving the system. This is somewhat more difficult to sell as a consultant as it requires a more sophisticated buyer, but is worth a lot of money if you can do it well
10 replies 19 reposts 262 likes 70K views View on X
cole murray Verified account @colemurray
you listened, right anon?
unusual_whales Verified account @unusual_whales 9h
BREAKING: Google says “the era of autonomous attacks has arrived.”

Hackers are now using AI to scan software updates, figure out which security flaw each patch fixes and build an attack before companies can install the fix, all without a human involved, according to a new report from the Google Threat Intelligence Group.

“Defending against machine-speed threats at human speed isn’t feasible,” Google said.
View quoted post on X
15 replies 19 reposts 681 likes 138K views View on X
cole murray Verified account @colemurray
im not convinced the idea of giving every agent chat its own sqlite db is a good idea

it creates fragmentation in the storage layer and complexity around standard db ops.

imagine you have 10MM agent chats across your service, and thus 10MM independent sqlite dbs.

user can return to an old session at any given time, so the dbs need to be persisted indefinitely.

schema changes means either doing a migration across millions of dbs or maintaining backwards compatibility with old versions, potentially for years.

analytics and search are now across 10MM dbs, requiring a way to aggregate and query across the data.

you could setup some kind of event stream, but now need to guarantee that each change makes it into the stream. you now get fun consistency and recovery problems.

idk

doesn’t feel like the right approach and significantly more complicated
40 replies 3 reposts 177 likes 21K views View on X