In the recent DEF CON Meet, technical director of security shop Endgame Hyrum Anderson disclosed a research paper on adapting API frameworks in building a malware that AV engines cannot identify.
The core function of the system is to build a legitimate looking app by making minor changes to the original app that can avoid AV detections. By making minor changes they have deceived the AV Engines and they named as gym-malware.
In a span of 15 hours of training, the software ran over 100,000 samples past an unnamed security engine. Around 60% of samples produced can bypass the system security.
Code for OpenAI Gym published in Github by Anderson and their team.
OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. This makes it possible to write agents that learn to manipulate PE files (e.g., malware) to achieve some objective (e.g., bypass AV) based on a reward provided by taking specific manipulation actions.