John Harnisher

Fascinating research on making language models more adaptable and efficient! 🧠💻 I recently came across an insightful interview with Kyle O'Brien, an Applied Scientist at Microsoft, discussing his team's paper on "Composable Interventions for Language Models." Key takeaways: Post-training interventions like model editing, unlearning, and compression can address issues in LLMs without full retraining The study explores how well these interventions work together - their "composability" Findings suggest certain orders of applying interventions work better than others More research is needed on how these techniques scale with larger models This work could significantly impact how we deploy and maintain AI systems in practice. It's exciting to see researchers tackling these practical challenges in AI development! #ArtificialIntelligence #MachineLearning #AIResearch #LanguageModels https://lnkd.in/eBBU7_BX

3

Interesting podcast with Christopher Manning, the Thomas M. Siebel professor in Machine Learning at Stanford University. It explores professor Manning’s perspectives on the intersection of linguistics and large language models, their ability to learn human language structures, and their potential to teach us about human language acquisition. #LLMs #linguistics #ML #GloVe #transformer architecture #attention #embeddings

4

Check out the Artificial Intelligence (AI) News Roundup for the Week of December 23 presented by the Mississippi Artificial Intelligence Network (MAIN)—the nation's first and only statewide AI initiative! One of the objectives of MAIN is to raise awareness about AI. MAIN helps facilitate this awareness by staying up-to-date on AI news. As part of this effort, MAIN shares daily AI news and compiles a weekly roundup featuring a variety of AI-related updates. Stay connected with MAIN and learn about FREE AI resources available to anyone in Mississippi: https://mainms.org #MAIN #Mississippi #MississippiAI #MississippiArtificialIntelligenceNetwork #AI #News #LatestNews #NewsUpdate #ArtificialIntelligence #AINews #Workforce #WorkforceDevelopment #GenerativeAI #GenAI

16

Adrienne Murphy discusses an outstanding example of data leadership by the Massachusetts Department of Early Education and Care. Transcript: Adrienne Murphy (Massachusetts Department of Early Education and Care): First, in terms of understanding the pre-K landscape, the CUSP network has really helped us understand, in this three and four-year-old space, who's being served across the mixed delivery system, which has been especially important when our governor, the Healey Administration, announced a new Gateway to Pre-K Initiative, which included a proposal to deliver universal, high-quality preschool access for four-year-olds in all of our Gateway cities, which are our sort of mid-sized urban centers that are sort of the regional economic anchors across the state. And the cost model has been able to help us to benchmark sort of where are we now, who's being served across our licensed and funded early education care providers, across our public K-12 system, across our private school system, to be able to understand sort of the full landscape of who we're currently serving. And, by virtue of who we're currently serving, also, who are we not serving? Sort of what's the delta between where we are now in pre-K access for four-year-olds across the state, but really particularly in these gateway cities? And then sort of, what's the delta between where we are now and the universal access that is proposed in the gateway to Pre-K initiative? So, this is, you know, really one instance in which being able to know who are we currently serving, who are we not serving, and where are we not serving children has become really key for us and for Massachusetts as sort of the policy design process. Click here to learn more: https://team3si.com/ #communityofpractice #dataforgood #earlychildhooddata #acf #ecids

1

Join Carnegie Mellon University's Philip Koopman on Sept 11, 10 AM EDT, for insights on leveraging #AI effectively. Learn about: • Classification & generative outputs • Addressing bias & safety • The 90/10 principle • Real-world Machine Learning applications Understand AI's potential and limitations. Register now! https://lnkd.in/gvhVk33y #MachineLearning #TechInnovation #BOSS2024 #ArtificialIntelligence #DeepLearning #Semiconductor #ML

8

Researchers at CDS have unveiled that transformer language models can solve complex problems without clear intermediate reasoning, challenging existing assumptions about their operation. Led by CDS PhD students Jacob Pfau and William Merrill, and CDS Associate Professor of Linguistics and Data Science Sam Bowman, this study highlights the use of nonsensical filler tokens that achieve perfect accuracy in specific tasks, raising concerns about  interpretability and accountability in LLMs. Explore the full findings: https://lnkd.in/eERzzKsU

13

Mathematics Education Article from #JCMM Volume 3, Issue 1! We are highlighting a notable article from a previous issue of the Journal of Computers, Mechanical and Management (JCMM): "The Study of the Value of π Probability Sampling by Testing Hypothesis and Experimentally" 📝 #Authors: Sanjay Kulkarni and Sandeep Kulkarni 🔍 #Abstract: This study evaluates the value of π using the Monte Carlo Simulation Method, comparing the results with #experimental values. It examines the distribution of results using non-parametric tests, providing valuable insights into #simulation techniques in the physical #sciences . 📖 Read the full @article here: https://lnkd.in/ghRKJehi Explore this insightful research and stay updated with JCMM for more groundbreaking studies. #Research #JCMM #MonteCarloSimulation #ProbabilitySampling #Engineering #Management #Mathematics

4

Benchmarking Large Language Models: Challenges and Future Directions 📊 In our latest blog post, Giancarlo Mori explores the challenges of establishing benchmarks for large language models (LLMs) and measuring their performance. Key points: 1️⃣ Diversity of tasks and the dynamic nature of language make comprehensive benchmarking difficult 2️⃣ Current evaluation metrics have limitations in capturing nuanced aspects of language 3️⃣ Addressing bias and ensuring scalability are crucial for effective benchmarking Read the full article for insights on future directions and recommendations for improving LLM benchmarking! https://lnkd.in/dNjvZfVz #MVYL #EnterpriseAI #LLM #AI #Substack

1

1 Comment

If you use large language models (LLMs) in your research, register for this seminar to hear Elucidata’s Dr. Abhishek Jha discuss how data quality impacts LLM performance. A reliable foundation that is well annotated and accessible to an LLM plays a major role in the value of its results. Using cancer research as an example, you’ll see examples of how LLM-powered artificial intelligence (AI) agents query across three versions of the same gene expression corpus with differing results. #medaff #datascience #webinar #AI https://buff.ly/4b4Jfvv

2

Checkout my recent Forbes article on counting the cost of an AI deployment.

22

1 Comment

The latest report by BCG on AI implementations finds that majority (98%) of companies are experimenting with Gen AI, but only a minority (26%) have the capabilities to implement it at scale, and even a smaller subset (4%) have done so to achieve results. Which category is your organization in? Leaders in AI find value in both core business processes and support functions; they are more ambitious with AI, focusing on fewer, high-impact projects that promise higher returns. They invest strategically, integrating AI into both cost reduction and revenue-generating activities. The report stresses that success in AI relies more on investing in people and processes than on technology alone. Common obstacles include prioritizing AI initiatives, integrating AI with existing IT systems, and addressing workforce readiness. Source: https://lnkd.in/eRj4BuVf

16

Harvard’s Kennedy School Shorenstein’s Center released a document that can help make AI systems more transparent. The report introduces the CLeAR (Comparable, Legible, Actionable, and Robust) Documentation Framework that outlines important guiding principles for AI documentation. This is could be an important strategy for helping to address the problem of AI systems’ perceptions as “black boxes,” where people don’t fully understand how their algorithms work or the origins of their data sources. Note that the document is only a discussion paper, but it still highlights some interesting concepts.

9

1 Comment

Read 2023 #MostRead paper of TAAS, "Modeling, Replicating, and Predicting Human Behavior: A Survey." This article provides a succinct yet systematic review of important approaches in two areas dealing with quantitative models of human behaviors. https://bit.ly/3UzrFKR Andrew Fuchs, Andrea Passarella, Marco Conti #machinelearning #openaccess

3

NEWS - Validation and Evaluation for Trustworthy Artificial Intelligence (VET AI) Act Introduced U.S. Senators John Hickenlooper and Shelley Moore Capito introduced the bipartisan Validation and Evaluation for Trustworthy Artificial Intelligence (VET AI) Act (details below). I'm excited about this bill because it builds on recommendations we--as the IEEE USA AI Policy committee--made in a position statement released last February (https://lnkd.in/e9sETwNM). It was a pleasure to co-lead this effort with Dr. Cari Miller. I'd also like to thank Sen. Hickenlooper's staff for being receptive to feedback that I, and colleagues, provided on this bill. Details: ➡ Directs NIST, in coordination with the Department of Energy and National Science Foundation, to develop voluntary specifications and guidelines for developers and deployers of AI systems to conduct internal assurance and work with third parties on external assurance regarding the verification and red-teaming of AI systems. ➡ Such specifications require considerations for data privacy protections, mitigations against potential harms to individuals from an AI system, dataset quality, and governance and communications processes of a developer or deployer throughout the AI systems’ development lifecycles. ➡ Establish a collaborative Advisory Committee to review and recommend criteria for individuals or organizations seeking to obtain certification of their ability to conduct internal or external assurance for AI systems. ➡ Require NIST to conduct a study examining various aspects of the ecosystem of AI assurance, including the current capabilities and methodologies used, facilities or resources needed, and overall market demand for internal and external AI assurance.

82

8 Comments

While current focus is largely on making #AI better at human-like search, #eRAG proposes that AI will likely become the primary search engine user, requiring fundamentally different approaches. Computer scientists at the University of Massachusetts Amherst have published (👉 https://lnkd.in/eAAKXh9N) and released ( 👉 https://lnkd.in/eymSeq24) a system for evaluating the reliability of AI-generated searches. eRAG, is a way of putting AI and search engines in conversation with each other, then evaluating the quality of search engines for AI use. At its core, eRAG is trying to solve a fundamental problem in AI-powered search: How do we effectively evaluate whether retrieved documents are actually useful for an AI's task? The brilliance of this approach is in its simplicity, instead of relying on human judgment or simple keyword matching, eRAG lets the AI itself be the judge of what's useful by testing each document's individual contribution to the task. The paper shows this works consistently better than previous methods, with significant correlation improvements (0.168 to 0.494 in Kendall's τ) while using 50x less GPU memory. The key advantage is its focus on its practical application - does this document help the LLM do its job? - rather than some abstract notion of relevance. Ultimately, the best way to evaluate if a retrieved document is useful for an AI is to simply test how well the AI performs with that document - a practical solution that mirrors how we might evaluate reference materials for human use, but optimized for #artificialintelligence. #aipatterns #emergingAI #RAG #enterpriseai

5

Faster, more accurate, less data needed = BaseModel.AI

5

I see a lot of innovation potential in using LLMs to build data products. Note that you also have to understand the limitations of GenAI in building viable data products. Not only is LLMs, such as ChatGpt , getting better and better, but they are also getting more data from partners such as FT. Using these GenAI to create your data products can be a worthwhile consideration. #dataproducts #genai #madt #innovation https://lnkd.in/e699SFYZ

5

While there are many concerns about AI for CXOs, they often merge into one: how will AI impact or change their initiatives? Join Gartner's complimentary leadership webinar to explore how AI and GenAI are reshaping the top priorities for leaders across the C-suite. #GartnerIT #AI #CXO #Webinar

6

1 Comment