Twitxy
Saved
@sercanarik's profile picture
View on X

Sercan Arık

@sercanarik

Artificial Intelligence @Google. Former Stanford PhD. Cultural, interdisciplinary enthusiast. I tweet about things that interest me.
San Francisco, CA sercanarik.com Joined April 2009
493 Following 705 Followers 767 Posts

Latest posts 5

Sercan Arık @sercanarik
Selective prediction can improve reliability of LLMs by allowing them to abstain from making predictions when they are unsure. Our novel framework, ASPIRE, is based on adaptation with self evaluation to push state-of-the-art in selective prediction. For details:
Google AI Verified account @GoogleAI Jan 18, 2024
Introducing ASPIRE, a framework that enhances the selective prediction capabilities of large language models, enabling them to output an answer paired with a confidence score. Learn how it outperforms state-of-the-art methods on a variety of QA datasets. → goo.gle/3RWpChi
Implementation of the ASPIRE framework via soft prompt tuning. We first generate the answer to the question with the first soft prompt and then compute the learned self-evaluation score with the second soft prompt.
View quoted post on X
1 replies 0 reposts 5 likes 1.7K views View on X
Sercan Arık @sercanarik
Prompting can be very important to get accurate outputs with LLMs but challenging for humans - handcrafting even a small number of demos can be difficult or, for unseen tasks, impossible. Our recent work on automatic prompting addresses this...
Google AI Verified account @GoogleAI Nov 2, 2023
Introducing a new approach for adaptive prompting of #LLMs that train with unlabeled samples + pseudo-demonstrations generated by the model itself to close the gap between few-shot and 0-shot performance on reasoning, NLU and language generation tasks. → goo.gle/3tY00IB
Illustration of USP in exemplary tasks (classification, QA and text summarization). The LLM first generates predictions on an unlabeled dataset whose outputs are scored with logit entropy, consistency or alignment, depending on the task type, and pseudo-demonstrations are selected from these input-output pairs. In Stage 2, the test instances are augmented with pseudo-demos for prediction.
View quoted post on X
Sercan Arık @sercanarik
Our recent work on using LLMs to query databases using natural language:
elvis Verified account @omarsar0 Jun 2, 2023
State-of-the-art Text-to-SQL with SQL-PaLM

- proposes SQL-PaLM, an LLM-based Text-to-SQL adopted from PaLM-2

- achieves SoTA in both in-context learning and fine-tuning settings

- incredible that the few-shot model outperforms the previous fine-tuned SoTA by 3.8% on the Spider benchmark

- few-shot SQL-PaLM also outperforms few-shot GPT-4 by 9.9%, using a simple prompting approach

arxiv.org/abs/2306.00739
View quoted post on X
Sercan Arık @sercanarik
Privacy concerns can arise as a key bottleneck for data sharing, especially for sensitive domains like healthcare.
We propose a novel generative modeling framework, that yields privacy-preserving synthetic EHR data with high fidelity.
Google AI Verified account @GoogleAI Dec 21, 2022
Analyzing electronic health records (EHRs) has great potential, e.g. to enhance patient care, but common anonymization methods can decrease the data’s utility. To that end, read how EHR-Safe generates high-fidelity & privacy-preserving synthetic EHR data→ goo.gle/3HZfpxn
Block diagram of EHR-Safe framework.
View quoted post on X
1 replies 0 reposts 11 likes 1.4K views View on X