The discussion about AI production, creation, and human/AI interaction seems to never stop. When a more digital world arrives, how will AI evolve as important productivity? AI technology company & blockchain game company rct AI recently published a new DRL model based on training AI in Axie Infinity to achieve the best performance of AI in blockchain games. This RL-based AI algorithm for large-scale action space, ACAR, has achieved a double improvement in efficiency and winning rate in a large number of simulated battle data, which has surpassed real player’s level. Some even call it Alpha Go in blockchain games. The emergence of ACAR (Action Clustering using Action Representation) will upgrade or innovate the use space and development direction of AI in blockchain games. It will provide more research and application directions for the improvement of man vs machine battle computing power, the construction of an immersive virtual world, and the deep interaction of users. The paper on the ACAR algorithm of rct AI – “Towards Modern Card Games with Large-Scale Action Spaces Through Action Representation” was selected for the IEEE Conference on Games, CoG 2022, and was invited to give an oral presentation. About the new DRL model In the last few years, the advent of AlphaGo and AlphaGoZero has made people realize that reinforcement learning is an algorithm that can perform as well as or even better than humans in games of chance. Traditional card and chess games have always been the classic direction of AI research. Popular topics include Texas Hold’em, Mahjong, and Doudizhu. These games all contain the intractable problem of “huge and complex action space”. In addition, AI also needs to find out the most beneficial strategy for itself in the current game based on the opponent’s game strategy and play style. rct AI has always insisted on “Future web3 with AI”, and it brings the AI battle research of card games to the blockchain game directly. This time, rct AI chose the most popular card battle blockchain game Axie Infinity as the research object, because it not only contains the above-mentioned “huge and complex action space” (the action space combination of all cards is around 10^23), and the nature of the game of chance, but also contains a huge number of card groups. (There are more than ten popular card groups in the whole game, at least dozens of card groups in total, and hundreds of cards with different effects). Since it is impossible to add prior knowledge of human strategies to all decks before training, the opponent’s card groups will not be the same in different games. This adds another challenge to the learning of the DRL (Deep Reinforcement Learning) model. In continuous experiments, rct AI proposes a more efficient and general RL-based AI algorithm ACAR to solve the above problems. First of all, the team introduced a pre-trained embedding function to solve the huge action space in this problem. Through the battle with different opponents, the final effect of the executed action is used to learn the representation of different combined actions to achieve the effect of efficiently exploring the action space. After having the embedding function for action representation, the team can not only use the evaluation network Q to evaluate the output actions in the Policy network in the subsequent RL training, but also use the embedding function to reduce the dimension of the current network output action, and then use Q to evaluate the adjacent actions of the reduced dimensionality action to select the optimal action in the current state. The RL training combines the Embedding function, and finally in a large number of simulated battle data, the rct AI algorithm not only outperforms the other two Baseline algorithms in most decks (the average winning rate is 5% and 7% higher), but also the response efficiency is also completely better than the other two algorithms (only 43% and 39% of the average time consumed in the other 2 algorithm). Application scenarios The launch of this research result has attracted widespread attention in the industry, opening up a new blue ocean for the application of AI in the field of web3 games. According to statistics from SupraOracles, the overall market value of web3 games has exceeded 40 billion US dollars. This research allows us to predict that, at least in blockchain games, AI’s participation in the economic cycle will bring true sustainability and stability, and completely liberate the liquidity of digital assets in the game. First of all, for game projects, the number, type, and combat power of game players will be expanded. In web3 itself, virtual identities and digital citizens cannot be avoided. rct AI has given it more capabilities that are comparable to or even surpass those of humans, allowing these AI-powered players to enrich the game ecosystem with high quality and efficiency, to help game projects achieve their goals at different stages in terms of user volume and types, project revenue, data retention, and product co-creation. Secondly, for players, under the current mainstream game financial system of “P2E”\”P&E”, a double upgrade of game experience and revenue will be achieved. On the one hand, users will have extensive and ever-changing gameplay, and will no longer face the trouble that other users’ levels are too different from their own. AI virtual humans can equally participate in various gameplays with human players, bringing players a more personalized intelligent interactive experience. On the other hand, AI-powered players can collaborate and divide labor with real players. While real players can deploy strategies, AI-powered players can yield farming, which will improve the efficiency of game battles, thereby providing the benefits of more stable automation projects. Previously, people in the industry were thinking, when a more digital world arrives, how will AI evolve as an important productive force, what role will it play, and will the interaction between humans and AI produce new variables? We have to admit that AI can help humans explore the potential of web3, and this is no longer…