Larry "Patch" Trowell
- The ATM security risks that banks, financial institutions, and the ATM industry face today
- Recent ATM security breaches, common attack vectors, and security vulnerabilities – from peripherals/communication attacks to backend network attacks
- Actionable ATM best practices to ensure your customers, their cash, and their data remain protected
On March 1, 2022, Larry Trowell was featured in a ChannelPro Network article titled, IAM for IoT. Preview the article below, or read the full article online here.
+ + +
IDENTITY AND ACCESS MANAGEMENT is hard enough when it’s mostly users you have to worry about. When large volumes of vulnerable IoT devices are involved as well, the challenges only get greater.
"IAM is already a complex subject, and the addition of IoT devices makes the entire process much more complex," says Larry Trowell, principal consultant at NetSPI, a penetration testing-as-a-service security company in Minneapolis.
In IT, IAM “is used to streamline user digital identities, and to enhance the security of user-facing front-end operations," says Dimitrios Pavlakis, a senior analyst at ABI Research. Policies for passwords, email, accounts, and more can be automated, like onboarding, to meet security requirements and compliance rules. These advantages apply to IoT devices as well as users, but there are numerous hurdles.
For instance, domain controllers used by many companies often have trouble supporting IoT devices with limited client intelligence, according to Trowell. Even cloud solutions prepared for IoT devices "may not be able to operate with the level of access businesses feel they should," he notes. Multiple IoT devices may need to maintain identities and roles between various accounts, leading to security gaps within this complex environment.
[post_title] => ChannelPro Network: IAM for IoT [post_excerpt] => On March 1, 2022, Larry Trowell was featured in a ChannelPro Network article titled, IAM for IoT. [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => channelpronetwork-iam-iot [to_ping] => [pinged] => [post_modified] => 2023-01-23 15:10:50 [post_modified_gmt] => 2023-01-23 21:10:50 [post_content_filtered] => [post_parent] => 0 [guid] => https://www.netspi.com/?p=27452 [menu_order] => 300 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [4] => WP_Post Object ( [ID] => 26933 [post_author] => 105 [post_date] => 2021-12-10 07:35:00 [post_date_gmt] => 2021-12-10 13:35:00 [post_content] =>On December 10, 2021, NetSPI Principal Security Consultant Larry Trowell was featured in an article written by David Marshall for VMBlog.com. Read the full article below or online here.
+++
With the holiday season in full swing, cybercriminals know consumers are relying heavily on online shopping to fulfill their Christmas gifting lists, and organizations are at an increased risk of threats. Here's some helpful advice from several cybersecurity experts.
NetSPI, Larry Trowell, Principal Consultant
"As we enter the holiday season, security professionals must be aware of the threats that come with holiday gifts, specifically smart IoT devices. These connected gadgets open up a new host of security risks for both employees' personal lives and corporate networks.
Over the last two years alone, more people have set up multiple devices that connect to a single home network, including corporate-issued computers and tablets. With so many devices already in play, employees need to understand that some of the most popular technology gifts, such as robot vacuums, Tile, and Alexa come equipped with Bluetooth and Wi-Fi, cameras and geo mapping. These capabilities create a complex system that is more prone to attacks because it has greater potential for flaws and vulnerabilities within an increased attack surface - especially when integrated with other home automation products.
In tandem with this increased tech adoption, the pandemic and rise in remote work brought all corporate devices into employees' homes and opened up Pandora's box for potential vulnerabilities -- home office networks are said to be 3.5 times more likely to be attacked than corporate networks. To better understand, assess, and manage how employees are accessing company networks during the holidays, companies should educate their workforce about potential risks to their home network that come with tech gifts, and set up regular tests of their corporate systems as computers leave the office. Having a security testing program set in place -- prior to the holidays -- can help to identify any vulnerabilities within the corporate network quickly and efficiently and allow employees to better understand all the risks at play this time of year."
Immersive Labs, Kevin Breen, Director of Cyber Threat Research
"Cyberattackers like to take advantage of human behaviors and the holiday season is no exception. The increase in online and in-store shopping makes for an easy in, whether via phishing emails that mirror holiday marketing campaigns or fraud through the digital domain.
Toys and gifts are also becoming more high-tech and connected to WiFi or Bluetooth. Sadly, manufacturers don't always consider the security risks when building these connected devices, since they're hyperfocused on the user experience, which can present some exposure to users.
The human element also makes the holidays a particularly vulnerable time. There's a societal pressure to exchange gifts, make memories, finish the year strong and make ends meet-creating a slew of open opportunities for cyber threats and disruption. We've seen some of the most impactful ransomware attacks happen during holiday periods, for example, where there's minimal security staffing and an increase in external commitments. Cyberattacks don't stop during the holidays, in fact, they're often amplified, so it's critical that organizations remain vigilant and prepared."
Gigamon, Joe Slowik, Senior Manager of Threat Intelligence
"Supply chains are especially vulnerable to cyber attacks this holiday season. Supply chain attacks raise the prospect of stealthy, nearly impossible to detect intrusions by subverting fundamental trusts between network operators and their suppliers, contractors, and related parties...
...While concrete proof or direct evidence for any of these alleged incidents is circumstantial at best and typically nonexistent, the nature of the problem makes proving (or disproving) such events difficult or impossible. Once fundamental system trust is questioned, discussion quickly shifts such that one must prove that a device is not compromised which is a near impossible task.
One mechanism for adversaries, defenders and networker owners to retain significant ‘first mover' advantage in that they own, manage, and (ideally) can design the landscape on which intruders must operate - emerges through implementing "zero trust" security architecture. One of the core mechanisms to achieve and maintain zero trust principles is rigorous network segmentation through physical and virtual mechanisms. System owners can reduce direct connectivity between devices and establish authentication or rigorous trust boundaries between segments. Adversary lateral movement then becomes significantly more difficult even if the initial breach takes place via a supply chain mechanism circumventing other controls. Thorough segmentation becomes especially valuable when paired with monitoring and visibility. System owners and network defenders gain insight into internal network traffic flows between discrete zones as opposed to just internal-external communications. Combined with a robust approach to C2 traffic monitoring described in the previous section, defenders gain layered visibility into adversary operations throughout multiple phases of operations."
Datto, Ryan Weeks, Chief Information Security Officer
"The holiday season presents a "perfect storm" of opportunity for threat actors. Timing is the sweet spot for most attackers; the longer it takes for someone to notice there has been an intrusion, the more damage they can do. With an abundance of shopping deals, marketing emails and greater online traffic, the holidays are a perfect time for employees to fall for phishing tactics that enable hackers to propagate throughout a network - long before a company even realizes it.
In fact, phishing emails top the list of successful attack vectors at 54%. Further, a lack of education, weak passwords and poor user practices are among the top causes for ransomware attacks. In the weeks leading up to the holidays, companies should ensure their employees are properly educated and trained on how to spot phishing tactics and thwart intrusions that could quickly spread to infect an entire organization during the holidays."
Veriff, Janer Gorohhov, Co-founder and Chief Product Officer
"The accelerated digital transformation of companies around the world has led to an increase in fraud rates globally, and retail is no exception. To combat this increase in fraud and maintain trust and safety online this holiday season, more organizations must leverage artificial intelligence tools to identify and stop bad actors in their tracks, saving online retailers money and protecting both their employees and customers."
[post_title] => VMBlog.com: Expert Commentary: Cybersecurity Threats During the Holidays [post_excerpt] => On December 10, 2021, NetSPI Principal Security Consultant Larry Trowell was featured in an article written by David Marshall for VMBlog.com. [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => vmblog-expert-commentary-cybersecurity-threats-during-the-holidays [to_ping] => [pinged] => [post_modified] => 2023-01-23 15:11:03 [post_modified_gmt] => 2023-01-23 21:11:03 [post_content_filtered] => [post_parent] => 0 [guid] => https://www.netspi.com/?p=26933 [menu_order] => 333 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [5] => WP_Post Object ( [ID] => 26721 [post_author] => 105 [post_date] => 2021-11-23 07:00:00 [post_date_gmt] => 2021-11-23 13:00:00 [post_content] =>While the best part of the holidays is spending time with family and friends, giving or receiving a new smart device can often be the icing on the cake for people of all ages. A recent Consumer Technology Association study noted that technology sales are expected to hit $142.5 billion this holiday season – a record high from the last few years.
However, with the pandemic creating a distributed workforce where employees log onto corporate networks at home, these fun holiday gifts may be the next big network security risk – for both employees’ personal lives and corporate networks.
Companies need to better prepare for security vulnerabilities associated with the holiday season, while more broadly achieving a better understanding of how personal and corporate networks are blending in the modern work environment. To prepare, employees need to be educated on the risks smart devices bring to their home networks and IT/security leaders need to bolster their systems to ensure they remain secure while employees work from home.
Understand the IoT security risks for remote workers
Network risks are evolving. Over the last two years alone, more people have set up multiple devices that connect to a single home network, including corporate-issued computers and tablets. With so many devices already in play, and more to come as gifts this holiday season, the attack surface has grown exponentially.
The problem with connected tech holiday gifts
Some of the most popular technology gifts come equipped with Bluetooth and Wi-Fi, cameras, and geo-mapping. A popular gift, Tile-like tracking devices meant to help consumers find everyday items that are easily lost, has created conversation and speculation within the security industry over the years. But the threat has heightened, as an Apple integration now allows these types of devices – including Apple’s own AirTag – to be added and tracked on its “Find Me” feature. If compromised, an outside party could begin tracking the user’s location without their permission and monitor living patterns to exploit the information and lure them into a phishing attack or other breach.
Additionally, through a partnership with Amazon, Tile can now integrate with Alexa and other Amazon devices to detect if a Tile is nearby. With this feature, malicious actors could find any Tile in the area, hack into its GPS functionality, track its location, and notify the Tile network of its name, location, etc. within a certain radius – opening home devices to potential exploitation.
The same threats lie within additional ‘smart’ gifts. Robot vacuums, regardless of the brand, are connected to home networks and also connect to the internet – and integrate with other home automation products. This extension of connectivity creates a complex system that is more prone to attacks because it has greater potential for flaws and vulnerabilities. When you integrate a camera onto these devices, the risks only grow. Threat actors could easily monitor users’ movements to understand daily patterns and even craft a blueprint of their home.
How to solve for these vulnerabilities at home
With so many new gadgets and technology gifts on the market, many main players in this space are not doing their diligence to ensure the proper security precautions are in place – especially since it's an unregulated industry and manufacturing companies often prioritize development over security to meet increasing demands. As you install more IoT devices and get new gadgets for the holidays, consider putting these devices on a guest network. This will separate your at-home devices from your corporate computer and technology, limiting the potential attack surface that malicious actors can exploit.
If an attack does arise, using a guest network makes it easier to track and pinpoint the exact location of the breach, while limiting the potential threat to your corporate or home network. It’s also a best practice to pick one home automation kit and standardize it across all the technology in your house so all items will seamlessly integrate into that one system.
Understand and prevent corporate network vulnerabilities
The transition to remote work, spurred by the pandemic, brought all corporate devices into employees' homes and opened up a can of worms for potential vulnerabilities – home office networks are said to be 3.5 times more likely to be attacked than corporate networks. Further, there is currently a misconception about which systems and software can securely switch between corporate and at home networks, meaning employees have potentially opened their corporate networks to security risks dating back to when they initially took their office gear home in Spring 2020. Knowing this, how can IT and security teams prepare for the holiday gift season, where even more tech will be added to the mix?
Audit your security tools early on
To better understand, assess, and manage how employees are accessing company networks during the holidays and to work from home, companies should set up regular tests of their systems. Having a security testing program set in place – prior to the holidays – can help to identify any vulnerabilities within the corporate network quickly and efficiently. It can also open IT and security teams' eyes to which devices are vulnerable when used at home on personal networks. It’s important for companies to understand where vulnerabilities lie and make sure their systems and devices are secure year-round, but even more so during the holidays when the majority of staff is working remotely or taking time off.
Educate!
Often, security breaches are caused by a general lack of awareness within employee bases. Corporations should develop mandatory training programs that bring potential vulnerabilities to light, and teach their workforce how to monitor, prevent and report potential dangers. Training programs should include a specific lesson on the dangers of smart devices and working on home networks – consider timing this specific training in late-November, early-December, before the holiday season kicks off.
The holidays should be a time where all employees can recharge and spend time with family, without worrying about work. The onus is on enterprises to prepare their workforce for potential IT threats, while also taking proactive measures to prevent potential vulnerabilities in their network. Smart tech gifts aren’t going away, but with proper protocols in place, IT teams, company leadership, and the broader employee network can all enjoy time off without the risk of a breach.
[post_title] => IoT: Great Holiday Gift or Network Security Nightmare? [post_excerpt] => Learn why IoT devices may be the next big security risk – for both employees’ personal lives and corporate networks. [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => iot-holiday-gift-or-network-security-nightmare [to_ping] => [pinged] => [post_modified] => 2022-12-16 10:51:42 [post_modified_gmt] => 2022-12-16 16:51:42 [post_content_filtered] => [post_parent] => 0 [guid] => https://www.netspi.com/?p=26721 [menu_order] => 342 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [6] => WP_Post Object ( [ID] => 26042 [post_author] => 105 [post_date] => 2021-07-27 07:00:10 [post_date_gmt] => 2021-07-27 12:00:10 [post_content] =>According to estimates from the ATM industry association (ATMIA), there are more than three million automated teller machines (ATMs) across the globe today, making it the most common method for consumers to interact physically with their bank. Since its inception in 1967, criminals have discovered many ways to hack into ATMs – and technological advancements have only made their efforts more lucrative.
Modern hackers aren’t solely after cash. Account numbers, pin numbers, debit card information, and points-of-entry to the internal network or firmware provider, can be accessed by exploiting known security vulnerabilities.
One particular cybersecurity threat banks must pay closer attention to is direct memory access (DMA) attacks. DMA attacks target the areas of a computer that require direct memory access, such as the PCI bus or USB/Thunderbolt ports. DMA attacks enable an adversary with physical access to a device to read and overwrite memory, giving them full control over the operating system (OS) kernel and the ability to perform malicious activity. Unfortunately, many devices exist today that have not addressed the concerns that DMA vulnerabilities revealed years ago.
In this blog post, I will explore common ATM security vulnerabilities and attack tactics and explain why DMA attacks require heightened awareness. Additionally, I will share best practices to implement to help strengthen your ATM cybersecurity efforts.
Common ATM security vulnerabilities
ATMs have a lengthy shelf life for an embedded device, often lasting 10 years before needing to be replaced. The ATMs are typically composed of a Windows Desktop PC provided by the ATM manufacturers (e.g. Diebold, NCR, Hyosung). The bank is then responsible for hardening the OS and ensuring updates and patches are applied to the system as needed.
Further, many ATMs that exist today still run on older more vulnerable versions of Windows. These systems can become very expensive to maintain and can take significant resources to properly protect. Keeping these systems secure becomes more difficult as they get older and require a lot of work to keep up with the latest attacks. When keeping these systems properly hardened, it’s easy to miss some potentially unrelated security measures – until it’s too late.
It’s not always an outdated OS itself that the attackers target. While there are zero-day vulnerabilities that exist, we often see the security risks within the bank’s custom ATM user interface and applications, a lack complete of system hardening, vulnerabilities in custom security protections from the vendor/manufacturer, or unencrypted communications over the USB. Here is a sampling of the top five common ATM attacks:
- Sensor Tampering/Forking Attacks: Tampering with the sensors to take out money from the ATM without it debiting the accounts. Example: Australia forking attack, 2014
- Black Box: Connecting an external device (“black box”) to the ATM’s cash dispenser, then using native commands to cause the machine to release currency, bypassing the need for a card or transaction authorization. Example: Diebold code theft, 2020
- Peripherals/Communication: Most of the important devices inside of a typical ATM are peripherals that communicate over USB and serial busses. There are some systems that have not properly implemented encryption over these media. This leads to attackers spying on the USB, replaying attacks, man in the middle attacks, or fuzzing the interfaces altering software. These attacks aren’t always limited to the bus lines, the peripherals and the systems that support them are also vulnerable to communication attacks. Example: NFC Replay attacks, HITBGSEC 2018 D2
- Malware/Jackpotting: An attacker finds some flaw in the system that allows them to install their own custom software to the ATM: via an insecure firmware update, a leaked/outdated certificate, or a flaw in the encryption. These attacks do not have to be against the ATM itself, some forms of ATM malware can be administered without physical access to the machine by leveraging a known exploit against a financial institution’s servers. The malware would then be passed to every ATM in the chain, compromising many machines in one strike. Examples of ATM malware families: Ploutus, Anunak/Carbanak, Cutlet Maker, SUCEFUL
- Direct Memory Access (DMA): DMA allows devices to directly communicate with the system’s memory by bypassing the OS and manipulating firmware. If exploited, adversaries can gain direct access to information and privileges. They often require physical access but can also be deployed remotely. Example: DMA attack, PCILeech USB3380
The risk of DMA vulnerabilities
Despite the security precautions hardware and software vendors have implemented, DMA attacks remain a reality for many enterprise devices today. DMA attacks began making waves mostly as theoretical attacks until video game hackers caught wind. In the video game space, DMA attacks allow players to bypass protections without triggering the anti-cheat software placed by game manufacturers – but the threat reaches much farther than one industry.
Any system where an attacker has physical access to the machine is vulnerable. And these attackers’ techniques have gotten much more covert over the years. DMA attacks have been gaining traction in the red team space and banks are shocked at how easy it is to bypass their ATM security using a single technique. Currently, not many banks are testing for DMA vulnerabilities today possibly due to the lack of awareness around this particular attack vector.
As mentioned earlier, DMA attacks may grant adversaries full control over the all the device’s memory, including the kernel as well as the entire OS. The problem with this access is that the memory is not, at least by default, segmented. This access is granted at the hardware level, and thus it can replace any area of memory it has access to... regardless of the privilege that memory is protected by.
DMA attacks have also evolved. One avenue of accessing the systems that have DMA access are PCIe cards. These cards are similar to the cards used for adding graphic cards to a PC but modified to communicate with outside controllers to give attackers access to the computer’s memory. These custom cards now are Wi-Fi enabled. This allows for attackers insert their attack hardware and leave. The attacker can then wait until a system is up and running, then at their leisure, draw secrets from the systems ram (encryption keys, pins, credit card numbers, etc.) or modify a running authenticated system to run shellcode dynamically placed into memory outside the purview of the most thorough antivirus or malware protection.
Remediating this issue is no easy task. ATMs employ a number of security precautions: hard drive encryption, firewalls, process monitoring software, etc. to ensure the system has not been modified. Unfortunately, DMA attacks can easily bypass these protections.
The best way to prevent ATM security attacks, like DMA attacks, is to strengthen your foundational cybersecurity efforts and gain a better understanding of your preparedness and the impact an attack on your devices would have. To help, here are six ATM security best practices to follow, beyond physically disabling the PCIe bus with epoxy.
6 ATM security best practices
- Disable hardware that isn’t supposed be in the system by default. Anything that is USB that is not used, disable. This includes thunderbolt adapters, storage devices, and USB ethernet adapters. Anything that increases the attack surface and isn’t needed should be removed.
- Ensure encryption is set up properly and confirm all links in the chain of encryption are followed. Make sure the encryption keys are kept safe. And, that communications between peripherals are encrypted as well.
- If the version of Windows used allows memory segmentation, enable it. For DMA vulnerabilities, if using windows 10, turn kernel DMA protection on.
- Ensure the Operating Systems are properly hardened.
- Limit the types of USB devices the ATM accepts and limit the value of the vendor ID (VID) and product ID (PID). For example, there is no reason for an external graphics card or an audio adapter to be accepted in the USB.
- Perform a penetration test of your ATM applications to gain a better understanding of the impact an ATM security incident or breach would have on your systems – and learn if your existing security controls are working as they’re supposed to.
The importance of ATM penetration testing services
Penetration testing services can tell you where your security is and, more importantly, where it is lacking. Pentesting can verify whether the ATM peripherals that handle sensitive data are properly encrypted or that encryption keys cannot be extracted from the firmware or the card reader. Is the encryption used to protect the hard drive strong enough or configured correctly? Is there any method that attackers can use to gain access to the keys – if so, what can they do once they have the keys?
Sometimes, it is not possible to prevent every attack. In these cases, you need to know what will happen once there is a breach and how well you are protected once a weakness is found. Then, make it as difficult as possible for an attacker to maneuver inside a system. Using outside pentesting teams is a great way to keep appraised of the latest attack methods and view your system from the perspective of an adversary.
The practitioners of machine learning were recently reminded that we are not isolated from security threats. Recently, the PyTorch-nightly dependency chain was compromised. According to its website, PyTorch is “an open source machine learning framework that accelerates the path from research prototyping to production deployment.” Meaning, if compromised by adversaries a baseline result could be running malware on their systems that would extract credentials and other sensitive information from the build computers, but it could have been much worse. Depending on the attack scenario, the AI models themselves could have been modified or extracted, which would result in various attack scenarios depending on the intended use of the model.
While machine learning seems like a new and exciting field, it isn’t a new discipline. Artificial intelligence (AI) has been a mathematical discipline since we figured out how to use a delta function to gradually decrease the error of a given equation. Yet, we have not put the emphasis on security like we have with other computer programming disciplines.
Machine learning is a new discipline and requires new strategies to secure, because like all software projects there are always vulnerabilities. So, how can one hack an artificial intelligence? Well there are a number of ways depending on the technique used to implement it.
Hacking AI IRL
Let's start with what happened recently. One of the major machine learning frameworks fell victim to what appears to be a type squatting attack. An attacker created a library that was similar or in some cases the same as a library that is called by an application, when the application, a python framework in this instance is updated, the repository pulls from the match that has the most priority which can be the most recently updated.
This resulted in a framework that was kept at the cutting edge of development, installing code that would copy protected information out of the OS that the framework was running as that could be used to create a breach into that application.
As bad as that is, it could be worse.
It could be possible to create an edge case or blind spot in an application built on a tampered framework that would affect its outcome during training. Imagine if some data set that the programmer intends to train the model on was provided by the injected code. It would be possible for any number of scenarios to occur, from market manipulation, self-driving cars not recognizing pedestrians who wore the wrong color shirt, to any number of weird scenarios. This leads me to another means to corrupt an AI.
Data Poisoning Attacks
There is an old programming adage that goes, “garbage in, garbage out” this is pretty much the thought process of a poisoning data attack. This is an attack in which an attacker corrupts the training data used to create an AI model, either during the initial or ongoing learning tracks.
For example, one of the most famous examples of this is Tai, a model created by Microsoft to handle natural language processing. However, it was designed to learn continuously, and a select few individuals decided to provide Tai with a dataset that consisted of offensive and biased language that made its responses reflect the data that it was trained with. Any AI that learns from information after it's been put into production can be poisoned. Examples would be security software that monitors standard behavior being slowly modified so that “unusual” tasks become “normal,” or self-driving cars that are programmed to adjust to sensors that degrade may learn to react poorly to some incoming sensor data.
If the data provided to a model isn’t poisoned, it can still be hijacked. The most common method is model inversion or invasion, in this instance an attacker would need to have gained access to the model itself, and developed an area of the model where the classification may not be as strong as desired and look for cases that may not be entirely correct. One method to accomplish this may be to fuzz the inputs of a model and look for changes, but a more effective method would be to implement an AI to attack the original; such a technique would be creating Generative Adversarial Networks to create models that are close to the original training set in key points but different enough that they should not be accepted if the model was as precise as was intended.
In some case it may be possible to reverse engineer the model, by viewing the activation weighs of each of the hidden layers in order to speed up the above attacks, but this is a complicated process and can be easily protected against by adding a layer of encryption to the completed model.
How to Protect ML and AI from Attacks
There are a couple of methods that can be used to protect machine learning and artificial intelligence projects that are reasonably effective.
- Protect your development environment: this includes your source code, your training sets, and filtering any data that gets added into your model.
- Use a diverse training set: Depending on the data used for the model, consider employing more diverse data sets. While GAN’s are primarily thought of to help produce realistic images and sounds, they do so by finding connections and pathways that trick other AI’s into accepting they are created by something other than an AI. If the GAN starts to develop a training set that deviates greatly from the preset training data, then it may introduce weird edge cases where unexpected results may result. Such as finding patterns where Masterprints and "Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition" and Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning are possible.
- Watermark or encrypt the model to make it harder for attackers to reverse engineer or clone.
- Risk assessment: Before deploying any machine learning model, it's important to assess the risks that the model may be targeted by and what risk it might pose — this includes identifying any potential attack vectors, and the likelihood/impact of a successful attack.
- Monitoring and logging: flag suspicious access and any data supplied to the model. Occasionally confirm that no data resulted in atypical results.
- Continual testing: Regularly testing the model to check that it hasn’t deviated in an unwarranted manner, check the hidden layers for areas that may be vulnerable to bad input. This can be done via fuzzing, visual analysis, removing layers and checking results, or just continuous evaluation of the model.
It's important to note that these steps may not be a one-time occurrence. Protecting any software project is a continuous process, even one that’s operation is sometimes referred to as black magic, as the threat landscape and the technology is constantly changing. Additionally, it's important to have a team that has knowledge in both the realms of AI and cybersecurity. The malicious mindset is invaluable to protecting assets such as this and even the most thorough engineer can miss some of the more devious “what-ifs.”
[post_title] => Think Outside the Prompt: Creative Approaches to Attacking AI [post_excerpt] => These fundamental steps help practitioners of machine learning (ML) secure ML and AI from non-prompted attacks. [post_status] => publish [comment_status] => closed [ping_status] => closed [post_password] => [post_name] => think-outside-the-prompt-creative-approaches-to-attacking-ai [to_ping] => [pinged] => [post_modified] => 2023-06-07 10:39:24 [post_modified_gmt] => 2023-06-07 15:39:24 [post_content_filtered] => [post_parent] => 0 [guid] => https://www.netspi.com/?p=30299 [menu_order] => 105 [post_type] => post [post_mime_type] => [comment_count] => 0 [filter] => raw ) [comment_count] => 0 [current_comment] => -1 [found_posts] => 7 [max_num_pages] => 0 [max_num_comment_pages] => 0 [is_single] => [is_preview] => [is_page] => [is_archive] => [is_date] => [is_year] => [is_month] => [is_day] => [is_time] => [is_author] => [is_category] => [is_tag] => [is_tax] => [is_search] => [is_feed] => [is_comment_feed] => [is_trackback] => [is_home] => 1 [is_privacy_policy] => [is_404] => [is_embed] => [is_paged] => [is_admin] => [is_attachment] => [is_singular] => [is_robots] => [is_favicon] => [is_posts_page] => [is_post_type_archive] => [query_vars_hash:WP_Query:private] => 6369bf7da48deaa0674b46b8837ac0a3 [query_vars_changed:WP_Query:private] => [thumbnails_cached] => [allow_query_attachment_by_filename:protected] => [stopwords:WP_Query:private] => [compat_fields:WP_Query:private] => Array ( [0] => query_vars_hash [1] => query_vars_changed ) [compat_methods:WP_Query:private] => Array ( [0] => init_query_flags [1] => parse_tax_query ) )