David Christian Aurel Meine

I am pleased to share that our paper on Jaynes machine is now available online via Open Access: https://lnkd.in/e64_Jgja John Hopfield and Geoffrey Hinton, who are receiving the Physics Nobel next month, looked to physics for inspiration to model neural networks. They developed the Hopfield network and the Boltzmann machine by using the Ising model and energy minimization as their guiding principles. I went in a completely different direction, looked to game theory, and identified potential games as the starting point. In our framework, the neurons and the connections compete in an error-minimization game to increase their effective utilities during training and reach an arbitrage equilibrium. The outcome is a lognormal distribution of weights and neuronal iota. Our paper shows that this is empirically observed in real-life deep neural nets such as the LLMs. The Hopfield model and the Boltzmann machine cannot predict the distribution of connection weights as it is beyond their capabilities. We show that the Hopfield network and the Boltzmann machine are special cases of the Jaynes machine. One can think of the former two as the "ideal gas law" version of neural networks, whereas ours is the "van der Waals equation". Just like the real-world gases are not ideal and behave more like van der Waals gases, the real-world deep neural networks such as VGGnet, BERT, and LLAMA are not Boltzmann machines, but are more like Jaynes machines.

132

14 Comments

🌟 Exploring ViSNet in the Recently Published AI2BMD Neural Network Potential 🌟 I recently came across the new AI2BMD neural network potential just published in Nature, and its underlying architecture—ViSNet—really stood out. ViSNet offers a unique approach to molecular modeling by efficiently capturing molecular geometry—bond lengths, bond angles, and dihedral angles—using innovative, low-complexity vector operations. This design strikes a strong balance between detail and efficiency, making it ideal for applications in molecular dynamics. 🚀 Central to ViSNet’s design is the Runtime Geometry Calculation (RGC) module, which dynamically extracts high-order geometric features as needed, avoiding explicit tensor storage. The key idea here is to break down complex geometric features into manageable vector representations. For instance, bond lengths are derived directly from the distance between atoms, while bond angles and dihedral angles are calculated from vector inner products and rejections between neighboring atoms. By summing unit vectors between atoms, RGC efficiently captures angular and torsional information at linear complexity, avoiding the O(N³) cost of traditional tensor operations. This keeps computational demands low and allows ViSNet to scale effectively to larger molecular systems, even when full geometric detail is required. 🧬 Vector-scalar interactive message-passing (ViS-MP) mechanism is another key method. Instead of treating scalar and vector features independently, ViSNet integrates them within each layer, letting vector features (like directions) and scalar features (like bond distances) inform each other. This combined representation captures detailed spatial information with fewer samples, enhancing data efficiency without sacrificing accuracy. Benchmarking on datasets like QM9, MD17, and Molecule3D, ViSNet demonstrated strong performance in predicting molecular properties such as energy and atomic forces, achieving high-fidelity results with as few as 950 training samples. 👍 The AI2BMD paper shows how ViSNet’s careful balance of accuracy, computational efficiency, and scalability can support realistic simulations, driving forward practical applications in drug discovery and beyond. 🌱 👉 Check out the ViSNet paper: https://lnkd.in/ggUZAeVg 👉 Check out the AI2BMD paper: https://lnkd.in/gxR6wtZX #DrugDiscovery #MolecularModeling #MachineLearning #NeuralNetworkPotential #AI4Science

32

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

23

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

25

For all large language models (LLM) can do, it's important to recognise what they can't. This research paper addresses a critical question: How reliable is LLM's knowledge? The authors, including the Head of Aveni Labs, Dr. Alexandra Birch, and Dr. Barry Haddow, our Head of Natural Language Processing, introduce a novel metric called MOdel kNowledge relIabiliTy scORe (MONITOR) to assess LLM's factual reliability. MONITOR works by asking models the same question in different ways and seeing how likely the model is to give the same answer each time. This innovative approach helps identify situations where an LLM might be making things up rather than providing facts. Read more: https://hubs.la/Q02w7fJk0 #LLM #largelanguagemodels #naturalLanguageProcessing #machinelearning #artificalintelligence

5

1 Comment

Thrilled to share a groundbreaking paper published by colleagues, Henrik Dreyer and Etienne Granet here at Quantinuum, featured in Nature's NPJ Quantum Information. This innovative research presents a novel method for simulating Hamiltonian dynamics on digital #quantumcomputers, bypassing the need for Trotterization – a standard but computationally expensive technique. Why is this important? The ability to simulate quantum systems more efficiently opens up new possibilities for quantum research and practical applications, especially in fields like #materialsscience, #chemistry, and #drugdiscovery, where understanding complex systems at the quantum level can lead to breakthroughs. Link to the article: https://lnkd.in/etS52Mer

63

Proprietary datasets and the combination of AI and experimental data are the key to build successful companies.

1

We recently held the BCI and WBE for Safe AI Workshop. Diogo Di Lucena's (AE Studio) talk reviewed 'Self Modeling in Neural Networks.' Di Lucena discussed how he and his collaborators implement self-modeling in neural networks, inspired by Graziano's attention schema theory, to reduce model complexity and foster prosocial behavior in multi-agent systems. By creating simplified models of one's attention and the exact mechanisms used to model others' attention, the team demonstrates that self-modeling reduces complexity, making it easier for agents to model the system and potentially enhancing prosocial behavior. Evaluating their method on various tasks and models, the authors find consistent results in complexity reduction, suggesting that their approach, grounded in neuroscience, is distinct from other regularization methods and applicable across diverse contexts to improve model complexity and prosocial behavior in multi-agent settings. Learn more: https://lnkd.in/gFtdec-m

2

👨‍🔬 🚀 Thrilled to announce the new paper, “Softening Ontological Reasoning with Large Language Models” co-authored by our scientists! 🔗 https://lnkd.in/drX-vKyQ 🔍 Why It Matters: Prometheux data platform enables highly optimised big data queries to fragmented data sources and schemas, showing exactly how it obtains results. 💡This work will advance Prometheux integration with LLMs, further complementing the precision of structured data-processing with the flexibility of language models, empowering organisations to make real-time, fully auditable decisions driven by both structured and unstructured data. #ExplainableAI #knowledgegraphs #AI #neurosymbolic #LLMs

41

5 Comments

Another great paper to highlight: Multiplexed quantum memory using cold atoms for quantum networking: https://lnkd.in/eWY6_gqJ We all know the importance of multiplexing both for high rate quantum networking and distributed quantum computing. Many platforms, especially the solid state ones, come in the play with the promise of large scale multiplexing but barely I see papers that actually do it. This recent paper on PRX is an experimental demonstration of a 144x multiplexed memory using Magneto Optically Trapped atoms and the setup is done so great to allow for a fully on demand quantum register operation. Of course the use of atoms have made this more feasible due to the natural indistinguishability they offer. They even go a step further to show the application of such register for entanglement synchronization. A real advantage of using Rb atoms is the possibility of transferring this scheme to a fully room temperature quantum memory. You might think room temperature memories are still far behind when it gets to entanglement storage but hopefully in a few weeks I can change your mind! For now though I hope you enjoy this beautiful work of science and engineering.

79

1 Comment

Keynote from Professor Neil Lawrence, Google DeepMind Professor of ML at University of Cambridge, as he presents a keynote: "The Atomic Human" at OpenUK Report Launch, drawing from his extensive research and new book. See the full schedule here https://lnkd.in/etBhY2W4 and watch the video recordings as they are shared. See Report https://lnkd.in/ewjeDY3j All content will be shared digitally after the event at https://lnkd.in/eCKkCQvt #openuk #opensource #theopenmanifesto #stateofopen #opensourceai #opendata #londondataweek #opensourcesoftware

6

🚀 Catch up on the recent advancements in Vision Language Models in our latest blog post. We cover early contrastive approaches like CLIP to more advanced models like Flamingo and LLaVA. 🔗 Link: https://lnkd.in/dbXnb7mX

14

1 Comment

I had a great time proving the XEB for approximate tensor representations and also working on the randomness extraction (q crypto aspect) of this Nature paper out of our group. https://lnkd.in/guPuxXph #quantum #publication #research #quantumsupremacy

63

2 Comments

Introducing a new short course: Function-Calling and Data Extraction with LLMs! Made in collaboration with Nexusflow and taught by Jiantao Jiao, Co-founder & CEO, and Venkat Srinivasan, Founding Engineer, this course focuses on two key skills for building LLM applications: function-calling and structured data extraction. Function-calling allows LLMs to execute external functions based on natural language instructions, while structured data extraction enables LLMs to retrieve useful information from unstructured text. 🛠️ Learn how function-calling is used 🤖 Use an LLM with multiple function calls 🌐 Build function calls that can access web services 📊 Use function-calling to extract structured data And more. Sign up for free: https://hubs.la/Q02CDGGZ0

744

19 Comments

Understanding and mitigating hallucinations in Large Language Models (LLMs) is critical for reliable AI applications. In a blog by Rogger Luo, we delve into advanced techniques to tackle this issue. Our experiments show promising results in reducing hallucination rates, offering valuable insights for those working with LLMs. Vectara’s RAG-as-a-service platform already helps mitigate hallucinations effectively, and we are excited to see ongoing research further enhancing these capabilities. Read the full blog here: https://lnkd.in/gjZ4yreb

13

3 Comments

Finally - achieving and surpassing the benchmarks that have long been held to mark a tipping point in quantum computing. Today's announcement, taken together with other disclosure in the past few weeks, extends our leadership in the quantum computing sector and brings reliable quantum computing out from the lab into the real world. This achievement should be one that all of us in the sector can enjoy and appreciate. I look forward to building on these developments as we get to the point where it becomes impossible for classical computers to simulate quantum processors. A few weeks ago we demonstrated our ability to scale our quantum computers. Then we disclosed stunning results with Microsoft on logical qubit development. And today we provide details of our world leading quantum volume numbers (exponentially better than literally anyone else) and for the first time, 2 qubit gate fidelities above '3 nines' across all our zones in our fully connected quantum processor. I could not be prouder of my colleagues. There is so much more to come as we continue to set the high bar in the industry and lead by example in exemplifying excellence. https://lnkd.in/esMQfmNt

227

17 Comments

SynDL: A Synthetic Test Collection Utilizing Large Language Models to Revolutionize Large-Scale Information Retrieval Evaluation and Relevance Assessment Researchers from University College London, University of Sheffield, Amazon, and Microsoft introduced a new test collection named SynDL. SynDL represents a significant advancement in the field of IR by leveraging LLMs to generate a large-scale synthetic dataset. This collection extends the existing TREC Deep Learning Tracks by incorporating over 1,900 test queries and generating 637,063 query-passage pairs for relevance assessment. The development process of SynDL involved aggregating initial queries from the five years of TREC Deep Learning Tracks, including 500 synthetic queries generated by GPT-4 and T5 models. These synthetic queries allow for a more extensive analysis of query-document relationships and provide a robust framework for evaluating the performance of retrieval systems. The core innovation of SynDL lies in its use of LLMs to annotate query-passage pairs with detailed relevance labels. Unlike previous collections, SynDL offers a deep and wide relevance assessment by associating each query with an average of 320 passages. This approach increases the scale of the evaluation and provides a more nuanced understanding of the relevance of each passage to a given query. SynDL effectively bridges the gap between human and machine-generated relevance judgments by leveraging LLMs’ advanced natural language comprehension capabilities. The use of GPT-4 for annotation has been particularly noteworthy, as it enables high granularity in labeling passages as irrelevant, related, highly relevant, or perfectly relevant..... Read our full take on this: https://lnkd.in/gATdzyyQ Paper: https://lnkd.in/gQGnGbGY

23

SVDQuant: A Novel 4-bit Post-Training Quantization Paradigm for Diffusion Models The rapid scaling of diffusion models has led to memory usage and latency challenges, hindering their deployment, particularly in resource-constrained environments. Such models have manifested impressive ability in rendering highly-fidelity images but are demanding in both memory and computation, which limits their availability in consumer-grade devices and applications that require low latencies. Therefore, these challenges need to be addressed to make it feasible to train large-scale diffusion models across a vast multiplicity of platforms in real time. Read the full article: https://lnkd.in/eMMcykZF Paper: https://lnkd.in/eYWr4jqS

8

"Learn Your Reference Model for Real Good Alignment," introduces Trust Region Direct Preference Optimization (TR-DPO). Developed by a visionary team led by Alexey Gorbatovski at Tinkoff, this paper pioneers a method that significantly advances the training of language models by updating the reference model during the training process itself. Why This Matters: ➡️ Enhanced Model Alignment: TR-DPO introduces dynamic updates to the reference model, which allow for more accurate and adaptive training compared to the traditional Direct Preference Optimization (DPO). ➡️ Improved Performance Metrics: Through rigorous testing, TR-DPO has demonstrated a 19% improvement in performance over DPO, making it a groundbreaking tool for achieving higher coherence, correctness, and detail in model outputs. Key Insights: ➡️ Dynamic Reference Model Updates: Unlike static methods, TR-DPO continuously adjusts the reference model during training, which keeps the model relevant and highly effective across changing data sets and conditions. ➡️ Human-Centric Results: The model notably excels in producing outputs that are coherent, detailed, helpful, and harmless—key qualities for any application from chatbots to complex decision-making systems. #analyticsvidhya #datascience #generativeai #researchpaper

16