RM新时代有限公司

Recently, CloudWalk Technology has made significant progress in visual large models. Its pedestrian foundation model stood out from Alibaba, Hitachi, and many other well-known universities, enterprises, and research institutions on four datasets—PA-100K, RAP V2, PETA, and HICO-DET—setting new world records.

Among them, the highest Fine-tuning accuracy on PA100K reached 92.89%, 5.2 percentage points higher than SOTA. The four datasets cover global human attributes (gender, age), local attributes (clothing style, accessories), carried attributes (phone, knife, handbag, etc.), and human-object interaction HOI (smoking, holding a knife, filming with a phone, etc.).

Table 1: CloudWalk Technology's Performance on PA-100K, RAP V2, and PETA Pedestrian Attribute Datasets

As the primary of the "six senses," vision accounts for over 70% of the information humans absorb from the outside world. The same applies to artificial intelligence. The pedestrian foundation model enables AI to recognize all information about people. The improvement in accuracy has very high practical value. This breakthrough means the technology has reached a level suitable for large-scale commercial use for the first time, and also signifies that computer vision has entered the "era of large models."

Multimodal Combined with Self-Supervised Learning: Building World-Leading Core Technology

Human-centric perception tasks have always been a research hotspot in the field of artificial intelligence. Large models have powerful representation capabilities and have been validated across multiple data modalities (such as language, audio, image, video, and vision-language). The pedestrian foundation model has become the foundation of visual large models. CloudWalk Technology, based on actual business deployment needs, has developed a human-centric pre-trained large model, focusing on various human-centric downstream tasks, practicing and implementing the concept of "human-machine collaboration."

Table 2: CloudWalk Technology's Performance on the HICO-DET Human-Object Interaction Dataset

CloudWalk's pedestrian foundation model uses over 2 billion data points, including a large amount of unlabeled datasets and image-text multimodal datasets. The richness and diversity of the datasets enable the model to extract very robust features, easily applicable to various pedestrian tasks.

Based on the self-supervised learning paradigm, CloudWalk Technology fully combines the advantages of contrastive learning and masked learning, enabling the model to contain rich semantic information while possessing strong texture detail extraction capabilities. To allow the model to learn richer pedestrian semantic information, a weakly supervised training paradigm is used in combination with multimodal data, further improving the model's performance.

In practical application scenarios, compared to specialized models designed for a single task, large models exhibit strong generalization, greatly reducing the need for real data, and even eliminating the need for additional real data collection. This significantly saves the cost of transferring to downstream tasks, allowing the capabilities of large models to be quickly migrated to new application scenarios. They can be widely applied in industries such as energy, transportation, manufacturing, and finance, and can create industry-specific large models for these sectors, providing deep empowerment.

Based on the Foundation Pre-trained Model: Significantly Reducing R&D Costs

Traditional pedestrian detection and analysis face many challenges, including a lack of scene understanding, resulting in only detecting all targets. Additionally, some pedestrian features are very subtle, requiring the model to have fine-grained recognition capabilities, and various large poses and environmental interferences can easily cause issues. Finally, the demand for pedestrian detection and analysis is variable and ever-changing, mostly requiring customized development. How to reduce development costs is crucial in the process of technology industrialization.

Based on the CongRong Large Model platform, developers can significantly reduce data dependency and improve training efficiency. Only 1% of the data volume is needed to achieve results close to those of customized models developed for specific scenarios, making it suitable for special industries where obtaining real samples is very costly.

Currently, this technology has been widely applied in safety monitoring at mines, construction sites, and special locations, monitoring workers' compliance with attire, posture, behavior, and other anomalies and violations.

Previously, CloudWalk Technology's cross-camera tracking technology broke world records three times consecutively from 2018 to 2020. The breakthrough in the pedestrian foundation model indicates that CloudWalk's CongRong Large Model has integrated resources such as the Western Computing Center, joint R&D centers, and industry data to create a superior AI visual large model, laying a solid foundation for the large-scale practice of the Human-Machine Collaborative Operating System and industry-specific large models in the field of industry intelligent upgrading.