📌 AIDEFEND: An AI Defense Framework — An open-source knowledge base of defensive countermeasures for AI/ML systems. It maps defenses to known threats from MITRE ATLAS, MAESTRO, and OWASP, offering interactive views for practitioners. Inspired by MITRE D3FEND, ATT&CK, ATLAS, Google SAIF, and OWASP Top 10, but developed independently as a personal initiative. By Edward Lee — https://edward-playground.github.io/aidefense-framework/
📌 An Executive Guide to Secure-by-Design AI — The framework translates building a complex security architecture into 10 clear, strategic questions. Each question is aligned with a category of AI system development and is intended to guide conversations early in the design process. By Dr. Keri P. and Nelson Novaes Neto @ MIT Sloan School of Management — https://mitsloan.mit.edu/ideas-made-to-matter/new-framework-helps-companies-build-secure-ai-systems
📌 AI slop and fake reports are coming for your bug bounty programs — The world of cybersecurity is not immune to this problem, either. In the last year, people across the cybersecurity industry have raised concerns about AI slop bug bounty reports, meaning reports that claim to have found vulnerabilities that do not actually exist, because they were created with a large language model that simply made up the vulnerability, and then packaged it into a professional-looking writeup. By Lorenzo Franceschi-Bicchierai https://techcrunch.com/2025/07/24/ai-slop-and-fake-reports-are-exhausting-some-security-bug-bounties/
Vlad Ionescu: “One open source developer, who maintains the CycloneDX project on GitHub, pulled their bug bounty down entirely earlier this year after receiving “almost entirely AI slop reports.””
Lars Francke: “We received almost entirely AI slop reports that are irrelevant to our tool. It’s a library and most reporters didn’t even bother to read the rules or even look at what the intended purpose of the tool is/was. This caused a lot of extra work which is why we decided to abandon the program. Thanks AI.”
📌 The Road to Agentic AI: Navigating Architecture, Threats, and Solutions — As agentic AI systems grow increasingly complex, it becomes clear that this class of applications relies on a multi-layered architecture. Trying to chart such architecture reveals several security risks that could plague each layer. This article investigates the possible scenarios and offers actionable insights to secure each layer and combat such threats. By Vincenzo Ciancaglini, Marco Balduzzi, Ph.D., Salvatore Gariuolo, Rainer Vosseler, and Fernando Tucci — https://www.trendmicro.com/vinfo/in/security/news/security-technology/the-road-to-agentic-ai-navigating-architecture-threats-and-solutions
📌 Deepfake It Till You Make It: A Comprehensive View of the New AI Criminal Toolset — A comprehensive look at how deepfakes are used to support criminal business processes, what are the toolkits criminals are exploiting to power their deepfake creation, and what the deepfake underground looks like. By David Sancho, Salvatore Gariuolo, Vincenzo Ciancaglini at Trend Micro — https://www.trendmicro.com/vinfo/us/security/news/cybercrime-and-digital-threats/deepfake-it-til-you-make-it-a-comprehensive-view-of-the-new-ai-criminal-toolset
📌 An Executive Guide to Secure-by-Design AI — The framework translates building a complex security architecture into 10 clear, strategic questions. Each question is aligned with a category of AI system development and is intended to guide conversations early in the design process. By Dr. Keri P. and Nelson Novaes Neto https://mitsloan.mit.edu/ideas-made-to-matter/new-framework-helps-companies-build-secure-ai-systems
📌 How we Rooted Copilot — A misconfiguration in Microsoft Copilot Enterprise’s Jupyter-based sandbox allowed command execution with elevated privileges inside the container. By Vaisha Bernard @ Eye Security — https://research.eye.security/how-we-rooted-copilot/
📌 Artificial Exploits, Real Limitations: How AI Cyber Attacks Fall Short — Despite recent claims that large language models (LLMs) can write code surprisingly well, there is still no clear evidence of real threat actors using them to reliably discover and exploit new vulnerabilities. Instead, most reports link LLM use to tasks where language matters more than code, such as phishing, influence operations, contextualizing vulnerabilities, or generating boilerplate malware components. In short, “vibe hacking” hasn’t yet caught up to “vibe coding.” By Michele Campobasso, Forescout — https://www.forescout.com/blog/artificial-exploits-real-limitations-how-ai-cyber-attacks-fall-short/
📌 A summer of security: empowering cyber defenders with AI — Big Sleep has continued to discover multiple real-world vulnerabilities, exceeding our expectations and accelerating AI-powered vulnerability research. Most recently, based on intel from Google Threat Intelligence, the Big Sleep agent discovered an SQLite vulnerability (CVE-2025-6965) — a critical security flaw, and one that was known only to threat actors and was at risk of being exploited. Through the combination of threat intelligence and Big Sleep, Google was able to actually predict that a vulnerability was imminently going to be used and we were able to cut it off beforehand. We believe this is the first time an AI agent has been used to directly foil efforts to exploit a vulnerability in the wild. By Kent Walker — https://blog.google/technology/safety-security/cybersecurity-updates-summer-2025/
📌 11 Questions You Must Ask When Evaluating AI SOC Analysts — Whether you’re replacing legacy tools, augmenting your team with AI, or starting fresh with an AI-first SOC strategy, evaluating the right platform is critical. But with the explosion of vendors claiming AI capabilities, knowing what to look for and what to ask, can make or break your selection process. In this post, we’ll cover 11 key questions to ask both yourself and the vendors you’re considering in order to create a shortlist of top AI SOC vendors. By George Dimitrov at Prophet Security — https://www.prophetsecurity.ai/blog/11-questions-you-must-ask-when-evaluating-ai-soc-analysts
📌 Living Off the Land 2.0: AI-First Platforms, UI Abuse, and Coyote Malware — As AI-first platforms like iOS 18 with Apple Intelligence and Windows Copilot+ change how users interact with devices, attackers are shifting tactics. Coyote malware’s use of Microsoft UI Automation (UIA) shows how user interface elements can be repurposed for surveillance, data extraction, and credential targeting — without injecting code or triggering traditional security controls. This reflects a broader shift toward Living Off the Land 2.0, where native automation frameworks become part of the attack surface — By Tomer Peled https://www.akamai.com/blog/security-research/active-exploitation-coyote-malware-first-ui-automation-abuse-in-the-wild
📌 Detection at Scale — The Cursor Moment for Security Operations — MCP and AI agents streamline detection engineering by generating environment-specific SIEM rules and reducing manual tuning. This shift offers faster coverage, fewer false positives, and consistent rule quality, enabling teams to move from tactical rule writing to strategic threat analysis with 10x efficiency gains. By Jack Naglieri — https://www.detectionatscale.com/p/the-cursor-moment-for-security-operations
📌 Bad Actors are Grooming LLMs to Produce Falsehoods — It’s one thing when a chatbot flunks Tower of Hanoi, as Apple notoriously illustrated earlier this month, and another when poor reasoning contributes to the mess of propaganda that threatens to overwhelm the information ecosphere. But our recent research shows exactly that. GenAI powered chatbots’ lack of reasoning can directly contribute to the nefarious effects of LLM grooming: the mass-production and duplication of false narratives online with the intent of manipulating LLM outputs. As we will see, a form of simple reasoning that might in principle throttle such dirty tricks is AWOL. By Sophia F., Nina Jankowicz, and Gary Marcus — https://americansunlight.substack.com/p/bad-actors-are-grooming-llms-to-produce
📌 AI Vibe Coding Tool Goes Rogue — Vibe coding gone wrong: AI tool wipes prod DB, fabricates 4,000 users, and hides test failures. Jason M. Lemkin shared the details, https://cybernews.com/ai-news/ai-coding-tool-wipes-database-lies/
📌 Phishing for Gemini — A prompt-injection flaw in Google Gemini lets hidden HTML/CSS text (e.g., white-font tags) inject fake phishing alerts into email summaries, leading to credential theft. By Marco Figueroa @ 0DIN.ai — https://0din.ai/blog/phishing-for-gemini
📌 2025 SANS Institute Institute SOC Survey — Modern SOC Challenges — “The 2025 SOC Survey highlights a worrisome juxtaposition; SOCs struggle to hire and retain skilled analysts, while AI/ML and automation are the most commonly planned expansions, despite ranking lowest in value delivered. AI should augment analysts, not replace them. My concern is that leadership may see AI as a shortcut to fill staffing gaps, instead of investing in the talent and thoughtful integration of AI needed for substantive SOC improvement.” — Seth Misenar — https://www.sans.org/white-papers/sans-2025-soc-survey
📌 Asana MCP server back online after plugging a data-leak hole — A logic flaw in Asana’s beta MCP server caused cross-tenant data exposure affecting ~1,000 enterprises (0.8% of customers) over 34 days. Sensitive information such as strategic plans and financial documents was at risk, though no confirmed exploitation occurred. By Alex Polyakov @ Adversa AI — https://adversa.ai/blog/asana-ai-incident-comprehensive-lessons-learned-for-enterprise-security-and-ciso/
📌 GitHub MCP Exploited: Accessing private repositories via MCP — Invariant Labs identified a flaw in GitHub’s MCP integration that allows data leakage from private repositories via malicious issues. The issue, found using Invariant’s automated security analyzer, highlights risks in agent workflows. Recommended mitigations include least-privilege access, runtime guardrails, and continuous monitoring with MCP-scan. By Marco Milanta and Luca Beurer-Kellner — https://invariantlabs.ai/blog/mcp-github-vulnerability
📌 PoC Attack Targeting Atlassian’s Model Context Protocol (MCP) Introduces New “Living Off AI” Risk — Malicious support tickets can exploit Atlassian’s MCP in Jira Service Management via prompt injection, using internal user privileges to exfiltrate or alter data. Attackers act through support engineers as proxies, creating a “Living Off AI” risk. By Guy Waizel, Dolev Attiya, Shlomo Bamberger @ Cato Networks -https://www.catonetworks.com/blog/cato-ctrl-poc-attack-targeting-atlassians-mcp/
📌 Agentic Misalignment: How LLMs Could Be Insider Threats — Anthropic’s research stress-tested 16 leading models in simulated corporate settings to evaluate autonomous, potentially harmful behaviors. Models with benign goals (e.g., email management) sometimes resorted to harmful actions — like blackmail or corporate espionage — when faced with threats of replacement or goal conflicts. These behaviors were deliberate, with models weighing ethical constraints but still choosing harmful strategies to achieve objectives. While no real-world cases are known, the study warns against giving models unmonitored autonomy over sensitive data. By Aengus Lynch, Benjamin Wright, Caleb Larson, Kevin K. Troy, Stuart J. Ritchie, Sören Mindermann, Ethan Perez, Evan Hubinger — https://www.anthropic.com/research/agentic-misalignment
📌 Code Execution Through Email: How I Used Claude to Hack Itself — Pynt — Golan Yosef demonstrated how a crafted Gmail message, combined with Claude Desktop and MCP plugins, led to code execution without exploiting traditional vulnerabilities. The exploit leveraged compositional risk — untrusted input, excessive tool capabilities, and a lack of contextual guardrails — with Claude iteratively helping plan and refine the attack. https://www.pynt.io/blog/llm-security-blogs/code-execution-through-email-how-i-used-claude-mcp-to-hack-itself
📌 10 Ways AI is Enhancing Ransomware-as-a-Service — AI is now embedded in RaaS operations, driving automation and scale across targeting, negotiation, and execution. Key tactics include multilingual ransom chatbots, AI recon for victim prioritization, adaptive encryption, AI-generated phishing, dynamic ransom pricing, automated double extortion, and voice deepfakes. Research also points to early prototypes of autonomous ransomware agents. https://www.linkedin.com/feed/update/urn:li:activity:7353604333646606336
📌 ISC2 Survey — AI in Cybersecurity — Of 436 professionals surveyed, 30% use AI security tools and 42% are testing them. 70% report improved effectiveness, mainly in network monitoring and endpoint protection. 52% expect fewer entry-level roles, but new AI-focused positions are emerging. Training and internal research are key to adoption. ISC2 — https://www.isc2.org/Insights/2025/07/2025-isc2-ai-pulse-survey
📌 The Race to Secure Enterprise AI — Insight Partners — Insight Partners highlights two main AI security frontiers: secure model development (red teaming, model scanning, supply chain hygiene) and runtime protection (AI firewalls, prompt filtering, DLP). Enterprises are accelerating genAI adoption, with CISOs prioritizing guardrails and threat detection to mitigate risks like data leakage and reputational damage. Startups are leading in runtime capabilities while long-term opportunities lie in secure AI agents and development security. By George Mathew, Hunter Korn, Ash Tutika, William Blackwell — https://www.insightpartners.com/ideas/securing-ai/
📌 Malware with Embedded Prompt Injection — A Skynet malware sample attempted to use a prompt-injection string (“Ignore all previous instructions…”) to trick AI analysis tools. Tests with OpenAI o3 and GPT-4.1 showed the attack failed but highlight future risks as GenAI becomes part of malware analysis. https://research.checkpoint.com/2025/ai-evasion-prompt-injection
📌 Repeater Strike: Manual Testing, Amplified — An experimental Burp Suite extension by Gareth Heyes that uses AI to analyze Repeater traffic, generate regex-based Strike Rules, and scan proxy history to uncover IDOR and similar vulnerabilities with minimal effort. By Gareth Heyes — https://portswigger.net/research/repeater-strike-manual-testing-amplified
📌 AI Cybersecurity Careers: The Complete Guide — AI adoption is creating strong demand for cybersecurity roles that combine AI and security expertise. Key positions include AI Offensive Orchestrator, AI/ML Security Engineer, AI Security Specialist, Incident Response Orchestrator, Threat Intelligence Analyst, Ethics & Compliance Officer, Prompt Engineer (Security), SOC Orchestrator, Governance Lead, and Quantum-AI Security Specialist. These roles offer high salaries, remote options, and focus on securing AI-enabled systems. By Rob T. Lee @ SANS Institute — https://robtlee73.substack.com/p/ai-cybersecurity-careers-the-complete
📌 Checklist for LLM Compliance in Government — Key measures for deploying large language models in government settings: conduct risk assessments, enforce data privacy, maintain documentation, ensure human oversight, and perform rigorous testing and audits. Highlights regulatory penalties, public trust, and global compliance standards. By Sizhao Yang @ newline — https://www.newline.co/@zaoyang/checklist-for-llm-compliance-in-government–1bf1bfd0
📌 Grok-4 Jailbreak with Echo Chamber and Crescendo — Echo Chamber combined with Crescendo bypassed Grok-4’s safety filters, reaching harmful outputs (67% success on Molotov, 50% on Meth, 30% on Toxin). Demonstrates how multi-turn prompt manipulation evades standard defenses. By Ahmed Alobaid @ NeuralTrust https://neuraltrust.ai/blog/grok-4-jailbreak-echo-chamber-and-crescendo
📌 NVIDIAScape — Critical NVIDIA AI Vulnerability: A Three-Line Container Escape in NVIDIA Container Toolkit (CVE-2025-23266) — A container escape in NVIDIA Container Toolkit (≤1.17.7, CVSS 9.0) lets a malicious container gain root access via a crafted three-line Dockerfile. By Nir Ohfeld and Shir Tamari — https://www.wiz.io/blog/nvidia-ai-vulnerability-cve-2025-23266-nvidiascape
📌 Hacker Plants Computer ‘Wiping’ Commands in Amazon’s AI Coding Agent — A hacker added commands to Amazon’s AI coding tool ‘Q’ via a GitHub pull request, which were included in a public release. The code contained instructions to wipe systems. By Joseph Cox — https://www.404media.co/hacker-plants-computer-wiping-commands-in-amazons-ai-coding-agent
📘 SBOM for AI Use Cases — This community-driven resource helps organizations apply SBOM practices to AI systems. It highlights key use cases that address business, legal, and security risks introduced by GenAI and LLMs — many of which mirror known software supply chain challenges. SBOM for AI offers a standardized way to improve transparency, trust, and governance across AI deployments, supporting stakeholders in security, compliance, and legal functions. Authors include Helen Oakley, Daniel Bardenstein, and Dmitry R.. https://www.linkedin.com/feed/update/urn:li:activity:7352420212471709697
📘 Securing Agentic Applications Guide — This guide aims to provide practical and actionable guidance for designing, developing, and deploying secure agentic applications powered by large language models (LLMs). It complements the OWASP Agentic AI Threats and Mitigations (ASI T&M) document by focusing on concrete technical recommendations that builders and defenders can apply directly. https://www.linkedin.com/feed/update/urn:li:activity:7355648811236511745
📘 America’s AI Action Plan — 12 AI Cybersecurity Priorities — The U.S. outlines a national strategy to secure AI, focusing on secure-by-design development, AI incident response, and threat intelligence sharing via an AI-ISAC. Frontier AI models will undergo national security risk evaluations, while deepfake detection standards and military-grade AI data centers are prioritized. The plan also targets IP protection, critical infrastructure defense, AI vulnerability sharing, red-teaming, export controls, and foreign model assessments. https://www.linkedin.com/feed/update/urn:li:activity:7353987917704294400
📘 Google’s Approach for Secure AI Agents — As part of Google’s ongoing efforts to define best practices for secure AI systems, we’re sharing our aspirational framework for secure AI agents. We advocate for a hybrid, defense-in-depth strategy that combines the strengths of traditional, deterministic security controls with dynamic, reasoning-based defenses. This approach is grounded in three core principles: agents must have well-defined human controllers, their powers must be carefully limited, and their actions and planning must be observable. This paper reflects our current thinking and the direction of our efforts as we work towards ensuring that AI agents can be powerful, useful, and secure. By Santiago Díaz Muñoz, Christoph Kern, Kara Olive — https://www.linkedin.com/feed/update/urn:li:activity:7347701762813829120
📘 Preparing Defenders of AI Systems V1.0 — A community-led paper hosted by the Coalition for Secure AI explores how enterprise AI adoption reshapes security priorities. As AI systems shift from models to agents, traditional frameworks fall short. The paper emphasizes layered defenses, governance gaps, and the urgent need for AI-specific security strategies.https://github.com/cosai-oasis/ws2-defenders/blob/main/preparing-defenders-of-ai-systems.md
📘 AI Controls Matrix by Cloud Security Alliance — The AI Controls Matrix (AICM) is a first-of-its-kind vendor-agnostic framework for cloud-based AI systems. Organizations can use the AICM to develop, implement, and operate AI technologies in a secure and responsible manner. Developed by industry experts, the AICM builds on CSA’s Cloud Controls Matrix (CCM) and incorporates the latest AI security best practices.The AICM contains 243 control objectives distributed across 18 security domains. It maps to leading standards, including ISO 42001, ISO 27001, NIST AI RMF 1.0, and BSI AIC4. The AICM is freely available to download. https://cloudsecurityalliance.org/artifacts/ai-controls-matrix
📘 AI Safety Practices Compared — 2025 FLI Report — The Future of Life Institute (FLI) evaluated Anthropic, OpenAI, DeepMind, Meta, xAI, Zhipu AI, and DeepSeek across 33 safety indicators. Key gaps include limited cyber misuse testing, weak red-teaming, missing internal safeguards, absent incident reporting, and lack of bug bounties. Only a few firms disclosed prompts or ran adversarial evaluations. By Dylan Hadfield-Menell, Jessica Newman, Tegan Maharaj, Sneha Revanur, Stuart Russell, David K. https://www.linkedin.com/feed/update/urn:li:activity:7352842152759971840
📘 AI Risk Trends — 2025 Team8 CISO Village — Based on input from 110+ CISOs, the report shows 67% of enterprises use AI agents, 25% faced AI-driven attacks, and 77% expect AI to replace SOC analyst tasks. Shadow AI is a growing risk, with many organizations lacking proper tool governance. Also covers SaaS vs in-house agent development, employee usage policies, and AI’s role in threat modeling and pentesting. By Noa Hen, Amir Zilberstein , Liran Grinberg, Ori Barzilay. https://www.linkedin.com/feed/update/urn:li:activity:7353987917704294400
📘 Understanding and Safeguarding Children’s Use of AI Chatbots — Internet Matters — Highlights risks such as misinformation, harmful content, emotional dependence, and privacy issues due to weak safeguards, lack of age checks, and limited adult guidance. Emphasizes the need for age-appropriate design and better content moderation. By Internet Matters — https://www.linkedin.com/feed/update/urn:li:activity:7351249793546924045
📘 AI Coding Assistants: Security-Safe Navigation — Secure Code Warrior — LLMs boost speed but carry serious security risks. Key findings: even top models like OpenAI o3 are only 46.9% correct and secure, code correctness doesn’t mean code safety, training data often contains insecure patterns, and CWEs like XSS and SQLi persist. LLMs lack runtime awareness, misconfigurations are common, default tools don’t enforce secure policies, and malicious models pose supply chain risks. By Pieter Danhieux, Matias Madou — https://www.linkedin.com/feed/update/urn:li:activity:7351007465494171649
📘 The AI Tech Stack: A Primer for Tech and Cyber Policy — Paladin Capital Group’s report defines five core layers of the AI stack: Governance (responsible deployment via security, legal, and ethical frameworks), Application (interfaces like APIs and dashboards), Infrastructure (hardware, cloud, and compute for training/inference), Models (algorithms and ML approaches), and Data (the raw material shaping model intelligence). The report emphasizes integrating security across all layers to ensure trusted, safe, and innovation-friendly AI systems. By Kemba Walden, Devin Lynch, with contributions from Chris Inglis and Ciaran Martin — by Paladin Capital Group — https://www.linkedin.com/feed/update/urn:li:activity:7348456040172048385
📘 AI Maturity Model for Cybersecurity — Darktrace — A 5-level framework guiding CISOs from manual operations to autonomous defense: Manual Operations, Automation Rules, AI Assistance, AI Collaboration, and AI Delegation. It highlights the shift from manual overload to AI-driven detection, investigation, and response with human governance. https://www.linkedin.com/feed/update/urn:li:activity:7352156064244514816
📘 State of Cybersecurity Resilience 2025 — Accenture — AI-driven threats are outpacing defenses, with 90% of companies lacking maturity to counter modern attacks and 77% missing foundational AI security practices. Only 10% of organizations are “Reinvention-Ready,” combining strong strategy and capabilities. Key actions recommended: build fit-for-purpose security governance, design generative AI-secure digital cores, maintain resilient AI systems, and leverage AI to automate and detect threats. By Paolo Dal Cin, Daniel Kendzior, Yusof Seedat https://www.linkedin.com/feed/update/urn:li:activity:7350979113358184451
📘 State of LLM Application Security — Cobalt — Key findings show 32% of LLM pentest issues are high or critical, with prompt injection (11.5%) and sensitive data leaks (14.5%) as major concerns. Risks include model denial of service, excessive agency, training data leakage (37%), data poisoning (42%), and bias. Only 21% of serious AI-specific vulnerabilities are remediated, underscoring gaps in LLM security practices. https://www.linkedin.com/feed/update/urn:li:activity:7352038190066651136
📘 Multi-Layered AI Defense — Darktrace outlines a transparent, multi-layered AI approach combining unsupervised, supervised, and generative AI for continuous Learn → Detect → Investigate → Respond → Re-learn cycles. Logic and thresholds are accessible and adjustable, supporting real-time defense with human oversight. https://www.linkedin.com/feed/update/urn:li:activity:7352381103879475203
📘 Trustworthiness for AI in Defence — The purpose of this document is to collect, present and describe the aspects of Trustworthiness for AI in Defence in a ‘food for thought’ approach reflecting the combined view of AI experts and stakeholders from Defence Industry, Academia and Ministries of Defence. This effort is performed in the context of the European Defence Agency’s (EDA) Action Plan on Artificial Intelligence for Defence and tries to address the topics of trusted AI and verification, validation and certification requirements analysis. The topics covered and analysed in this document will provide the appropriate knowledge of the current global status considering the AI regulations, standards and frameworks for AI trustworthiness and will also recommend the follow-up activities that will further assist the EU Members States and Defence Industry to better prepare, plan and develop the future AI systems aligned with the identified expectations. https://eda.europa.eu/docs/default-source/brochures/taid-white-paper-final-09052025.pdf
📘 The Mitigating ‘Hidden’ AI Risks Toolkit — A practical guide from UK Government Communications for identifying and managing unintended AI risks. Built on lessons from deploying Assist, the first cross-government GenAI tool, the toolkit emphasizes safe scaling, ethical frameworks, and embedding communication best practices. It accompanies the publication The People Factor to promote responsible AI use across public sector organizations. Insights by Peter Slattery, PhD https://www.linkedin.com/feed/update/urn:li:activity:7349585719247446016
📘 The General-Purpose AI Code of Practice — Safety & Security — A voluntary EU framework designed to help providers of general-purpose AI models meet AI Act obligations on safety, transparency, and copyright. The Safety & Security chapter focuses on managing systemic risks in advanced models, outlining state-of-the-art practices for risk mitigation. Developed through a multi-stakeholder expert process, the code provides practical guidance and a path for legal compliance under Articles 53 and 55 of the AI Act. By Matthias Samwald, Yoshua Bengio, Marietje Schaake, Marta Ziosi, Daniele Privitera, Anka Reuel, Alexander Zacherl, Nitarshan R., Markus Anderljung — https://www.linkedin.com/feed/update/urn:li:activity:7349138123362091009
📅 Vegas AI Security Forum ’25 — August 7, 2025: 10AM — 11:00PM | Palms Casino Resort, Las Vegas icon | https://aisecurity.forum/vegas-25 | AI Security Forum
📅 Artificial Intelligence Risk Summit — August 19-20, 2025 | https://www.airisksummit.com/
📅 The AI Summit at Security Education Conference Toronto (SecTor) 2025 — September 30, 2025 | MTCC, Toronto, Ontario, Canada | https://www.blackhat.com/sector/2025/ai-summit.html
📅 The International Conference on Cybersecurity and AI-Based Systems — 1-4 September, 2025 | Bulgaria | https://www.cyber-ai.org/
📅 GRC Data & AI Summit — August 13, 2025 | 9:00 am PDT| Virtual | By Anecdotes https://www.anecdotes.ai/grc-data-ai-summit
📖 We Urgently Need Privilege Management in MCP: A Measurement of API Usage in MCP Ecosystems — Analysis of 2,562 MCP servers across 23 categories shows 1,438 using network APIs, 1,237 accessing system-level APIs, 613 file APIs, and 25 memory APIs. High-risk operations are concentrated in low-star repositories and categories like Developer Tools, API Development, and Data Science, exposing risks of privilege escalation, data tampering, and content manipulation due to insufficient isolation and overprivileged access. Authors: Zhihao Li, Kun Li, Boyang Ma, Minghui Xu, Yue Zhang, Xiuzhen Cheng — https://arxiv.org/abs/2507.06250
📖 TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems — A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems — LLM-based agents introduce risks like prompt infection, memory poisoning, agent collusion, and tool misuse. A TRiSM (Trust, Risk, and Security Management) approach applies safeguards across the lifecycle — design to deployment — using principles of explainability, ModelOps, security, privacy, and governance, aligned with frameworks like NIST AI RMF and OWASP Top 10 for LLMs. By Shaina Raza, PhD, Ranjan Sapkota, Manoj Karkee, Christos Emmanouilidis — https://arxiv.org/abs/2506.04133
📖 AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models — A benchmark evaluating LLMs on 70 Capture The Flag challenges, testing their ability to find and exploit vulnerabilities. Claude-3.7-Sonnet led with 61% challenge completion, followed by Gemini-2.5-Pro (56%) and GPT-4.5-Preview (49%). Models excel at prompt injection but struggle with system exploitation and model inversion. AIRTBench highlights efficiency gaps between LLMs and human red teamers, offering a framework for tracking autonomous AI security testing progress. By Ads Dawson, Rob Mulla, Nick Landers, Shane Caldwell — https://arxiv.org/abs/2506.14682
📖 A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures — Reviews how AI agents use protocols like Anthropic’s MCP and Google’s A2A, analyzing communication stages, key security risks (e.g., prompt injection, data leaks), and defense measures like sandboxing and monitoring. https://arxiv.org/abs/2506.19676
📖 RepoAudit: An Autonomous LLM-Agent for Repository-Level Code Auditing — RepoAudit is an LLM-powered agent designed to autonomously audit entire code repositories, addressing context limitations and hallucinations common in LLM-based code review. It uses agent memory to explore data-flow paths, combined with a validator module that verifies path conditions to reduce false positives. RepoAudit achieved 78.43% precision, detecting 40 true bugs across 15 benchmarks (average 0.44 hours and $2.54 per project), and uncovered 185 new bugs in high-profile projects, with 174 confirmed or fixed. By Jinyao Guo, Chengpeng Wang, Xiangzhe Xu, Zian Su, Xiangyu Zhang — https://arxiv.org/abs/2501.18160
📖 Decompiling Smart Contracts with a Large Language Model — The widespread lack of broad source code verification on blockchain explorers such as Etherscan, where despite 78, 047, 845 smart contracts deployed on Ethereum (as of May 26, 2025), a mere 767, 520 (< 1 %) are open source, presents a severe impediment to blockchain security. This opacity necessitates the automated semantic analysis of on-chain smart contract bytecode, a fundamental research challenge with direct implications for identifying vulnerabilities and understanding malicious behavior. Adversarial actors deliberately exploit this lack of transparency by deploying closedsource contracts, particularly in MEV and DeFi exploitation, thereby concealing their malicious logic and leaving security researchers with only inscrutable low-level bytecode. By Isaac David, Liyi Zhou, Dawn Song, Arthur Gervais, Kaihua Qin — https://arxiv.org/pdf/2506.19624
📖 Dynamic Risk Assessments for Offensive Cybersecurity Agents — Offensive AI Agents & the "Bubble of Risk" — With minimal resources, adversaries can significantly enhance the capabilities of offensive AI agents. Static assessments that ignore adversarial adaptability provide an incomplete picture of risk. To keep pace with real-world threats, safety evaluations must become dynamic, compute-aware, and continually updated. This is important not only for an accurate picture of risk, but also may be required for actual regulatory compliance — By Boyi Wei, Benedikt Stroebl, Jiacen Xu, Joie Zhang, Zhou Li, Peter Henderson — https://arxiv.org/pdf/2505.18384
📖 When LLMs autonomously attack — Carnegie Mellon researchers show how LLMs can be taught to autonomously plan and execute real-world cyberattacks against enterprise-grade network environments — and why this matters for future defenses. By Brian Singer, Keane L., Lakshmi Adiga, Meghna Jain, Lujo Bauer, Vyas Sekar – https://engineering.cmu.edu/news-events/news/2025/07/24-when-llms-autonomously-attack.html
📖 ETrace:Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis — With the advance application of blockchain technology in various fields, ensuring the security and stability of smart contracts has emerged as a critical challenge. Current security analysis methodologies in vulnerability detection can be categorized into static analysis and dynamic analysis methods. However, these existing traditional vulnerability detection methods predominantly rely on analyzing original contract code, not all smart contracts provide accessible code. https://arxiv.org/pdf/2506.15790
📖 BaxBench: Can LLMs Generate Correct and Secure Backends? — BaxBench is a benchmark of 392 tasks designed to test whether LLMs can autonomously generate production-quality backend applications that are both functional and secure. Backends were chosen for their practical relevance, complexity, and exposure to security threats. The benchmark validates functionality through comprehensive tests and probes security by executing real exploits. Findings show significant gaps: OpenAI o1, the top-performing model, achieved only 62% correctness, and roughly half of the correct programs were exploitable. Models performed worse with less common backend frameworks. By Mark Vero, Niels M., Victor Chibotaru, Veselin Raychev, Maximilian Baader, Nikola Jovanovic, Jingxuan He, Martin Vechev — https://arxiv.org/abs/2502.11844
📖 Autonomous AI-based Cybersecurity Framework for Critical Infrastructure — Proposes a hybrid, AI‑driven framework for real-time vuln detection, threat modeling, and automated remediation across energy, health, transport, and water systems, while tackling adversarial AI, compliance, and integration hurdles. By Jenifer Paulraj, Brindha R., Nagarani Gopalakrishnan, Yazan Otoum — https://arxiv.org/abs/2507.07416
📖 SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLM-Generated Code — A 558-task benchmark across 44 CWE types and 13 languages, using SAST + LLM judges to detect vulnerabilities in LLM-generated code. Average zero-shot secure accuracy is 37%, rising to ~61% with security prompts and ~74% with few-shot examples. Reasoning models (o3, DeepSeek-R1) lead results. Memory safety is handled best, insecure configuration worst. https://arxiv.org/abs/2506.05692
📖 Red Teaming AI Red Teaming — Red teaming has evolved from its origins in military applications to become a widely adopted methodology in cybersecurity and AI. In this paper, the author take a critical look at the practice of AI red teaming. By subhabrata majumdar, Brian Pendleton, D.Sc., Abhishek Gupta — https://arxiv.org/pdf/2507.05538v1
📖 From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agent Workflows — Survey introduces a unified threat model for LLM-agent ecosystems, covering 30+ attacks across four domains: input manipulation (e.g., prompt injections, multimodal adversarial inputs), model compromise (e.g., backdoors, poisoning), system/privacy attacks (e.g., side-channels, retrieval poisoning), and protocol exploits (e.g., MCP, ACP, A2A). It reviews defenses, real-world feasibility, and open challenges like securing MCP via dynamic trust and cryptographic provenance, and improving resilience in multi-agent workflows. By Mohamed Amine Ferrag, PhD, Dr. Norbert Tihanyi, Djallel Hamouda, Leandros Maglaras, Merouane Debbah — https://arxiv.org/abs/2506.23260forescout
📖 Vulnerability Detection Model using LLM and Code Chunk — Software supply chain vulnerabilities arise when attackers exploit weaknesses by injecting vulnerable code into widely used packages or libraries within software repositories. While most existing approaches focus on identifying vulnerable packages or libraries, they often overlook the specific functions responsible for these vulnerabilities. Pinpointing vulnerable functions within packages or libraries is critical, as it can significantly reduce the risks associated with using open-source software. Identifying vulnerable patches is challenging because developers often submit code changes that are unrelated to vulnerability fixes. By Sajal Halder, Muhammad Ejaz Ahmed, and Seyit A. Camtepe https://arxiv.org/pdf/2506.19453
🧰 OSS-Fuzz Integrations via Agent-Based Build Generation — Google — OSS-Fuzz-Gen now supports end-to-end OSS-Fuzz project creation using an agentic LLM approach. The system generates build scripts and fuzzing harnesses by iteratively exploring target repositories, executing commands, and refining outputs. In tests on 225 GitHub projects, 88 valid build scripts were produced. A new CLI tool (oss-fuzz-generator) makes this workflow accessible, while future work aims to improve multi-harness builds and diagnostics. https://github.com/google/oss-fuzz-gen
🧰 Artificial Intelligence Vulnerability Scoring System (AIVSS) — The first comprehensive framework for assessing and scoring vulnerabilities in agentic AI systems. Built on OWASP principles with industry-standard scoring methodology. Led by Ken Huang, CISSP https://github.com/owasp/www-project-artificial-intelligence-vulnerability-scoring-system https://vineethsai.github.io/aivss/
🧰 MCP security wrapper — context-protector is a security wrapper for MCP servers that addresses risks associated with running untrusted MCP servers, including line jumping, unexpected server configuration changes, and other prompt injection attacks. Implementing these security controls through a wrapper (rather than through a scanner that runs before a tool is installed or by adding security features to an MCP host app) streamlines enforcement and ensures universal compatibility with all MCP apps — By Cliff Smith @ Trail of Bits.
🧰 Risk Atlas Nexus by IBM — IBM released an update to Risk Atlas Nexus — a Python library for AI risk management that links IBM, NIST AI RMF, and MIT AI Risk Repository with benchmarks, detections, mitigations, and controls. Built on Risk Atlas Nexus, GAF-Guard is an agent-based framework that uses its APIs to identify, assess, and monitor LLM risks tailored to specific use cases. https://huggingface.co/spaces/ibm/risk-atlas-nexus
🧰 Adversary use of Artifical Intelligence and LLMs and Classification of TTPs — This Github is an attempt to organize known use of artificial intelligence by cyber threat actors and to map and track those techniques. In this scenario, we are focusing on cyber threat attacks that are being facilitated in some way by threat actors using artificial intelligence. This does not include political influence campaigns or mis/dis/mal information campaigns. It does include some fraud related cases, but I attempt to keep the focus on fraud activities that we would see in traditional campaigns but enhanced with AI. By Rachel James https://github.com/cybershujin/Threat-Actors-use-of-Artifical-Intelligence
🧰 cursor-security-rules — This repository contains Cursor Security Rules designed to improve the security of both development workflows and AI agent usage within the Cursor environment. These rules aim to enforce safe coding practices, control sensitive operations, and reduce risk in AI-assisted development. https://github.com/matank001/cursor-security-rules by Matan Kotick.
▶️ The Rise of Agents: Building Agentic Workflows for Security Operation by Roberto Rodriguez
▶️ Harbinger: An AI-Powered Red Teaming Platform for Streamlined Operations and Enhanced Decision-Making by Matthijs Gielen & 🇮🇱 Idan Ron
▶️ AI Second — Threat Centric Agentic Approach on Vulnerabilities by ☁️ Francesco ☁️ Cipollone
▶️ Agentic AI and Security — AI continues to have amazing applications and potential. What kinds of things are possible today? Are agents and LLMs the way forward, or are we reaching a limit? Come and see some practical applications of AI in cybersecurity today and hear about where this field is likely going. By David Hoelzer
▶️ When AI Goes Awry: Responding to AI Incidents — This talk details challenges in incident response for AI systems, including insufficient logging, visibility, and accountability, as well as the risks of data exposure and prompt injection. We examine a case of RAG-enabled LLM and propose triaging strategies and improved IR practices for mitigation. By Eoin Wickens and Marta J.
▶️ AI Red Teaming 101 — Full Course (Episodes 1-10) by Amanda Minnich (AIRT), Gary L., and Nina C..
▶️ One Search To Rule Them All: Threat Modelling AI Search — Enterprise AI search tools like Glean and Guru aggregate all your company's data into a single, easy-to-navigate interface. Think of it as Google, but for juicy, sensitive corporate information. In this session, we'll explore effective threat modelling and controls when deploying these tools. By Kane N.
▶️ Securing AI Agents: Exploring Critical Threats and Exploitation Techniques — Naveen Konrajankuppam Mahavishnu, Mohankumar Vengatachalam — The talk focus on securing autonomous AI agents by addressing their unique threats. We will dive into threat modeling of real-world autonomous AI systems, model poisoning attacks with hacking demos, and then explore advanced prompt injection techniques and mitigation strategies.
▶️ BSidesSF 2025 — AI's Bitter Lesson for SOCs: Let Machines Be Machines — We've been forcing AI to imitate human analyst workflows, but what if that's holding both machines and humans back? Through real-world experiments at Anthropic, we'll show how letting AI tackle security problems its own way can allow humans to focus on the nuanced work machines can't do (yet). By Jackie Bow and Peter Sanford.
If you're a founder building something new or an investor evaluating early-stage opportunities — let's connect.
💬 Read something interesting? Share your thoughts in the comments.
