CN118820705A - Data processing method, device, storage medium, equipment and program product - Google Patents

Abstract

The application discloses a data processing method, a device, a storage medium, equipment and a program product, wherein the method comprises the following steps: acquiring initial input characteristics of data to be processed, wherein the initial input characteristics comprise object characteristics of a target object, resource characteristics of a plurality of resources to be recommended and scene characteristics, and the scene characteristics comprise scene identifications; performing feature compression processing on the initial input features to obtain low-dimensional input features; selecting a target sub-scene network matched with the scene identification from a plurality of sub-scene networks according to the scene identification in the scene characteristics, wherein each sub-scene network has independent scene parameters; performing feature processing on the low-dimensional input features based on the target scene-dividing network to obtain scene embedding features of the data to be processed; and recommending resources to the target object based on the initial input characteristics and the scene embedding characteristics. The method and the device can improve the calculation efficiency and optimize the scene adaptability, and improve the accuracy and individuation level of recommendation under different scenes.

Description

Data processing method, device, storage medium, equipment and program product

Technical Field

The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, storage medium, device and program product.

Background Art

In the current recommendation system technology, handling recommendation needs in different scenarios is a major challenge. Since different scenarios have different styles and contents, and users' usage habits and preferences are also different, specific modeling is required for different scenarios. Traditional scenario-based modeling methods include training a separate model for each scenario, training a unified model to process all scenario data, and adjusting scenario parameters through auxiliary structures under a common framework. Training multiple models separately is costly and difficult to maintain, while the unified model is easily affected by high-traffic scenario data, resulting in poor recommendation results in low-traffic scenarios. The auxiliary structure adjustment method still cannot completely eliminate the mutual influence between different scenario data.

Summary of the invention

The embodiments of the present application provide a data processing method, apparatus, storage medium, device, and program product, which can improve computing efficiency and optimize scene adaptability, and improve the accuracy and personalization level of recommendations in different scenarios.

On the one hand, an embodiment of the present application provides a data processing method, the method comprising:

Acquire initial input features of the data to be processed, wherein the initial input features include object features of a target object, resource features of a plurality of resources to be recommended, and scene features, wherein the scene features include a scene identifier;

Performing feature compression processing on the initial input features to obtain low-dimensional input features;

According to the scene identifier in the scene feature, a target sub-scene network matching the scene identifier is selected from a plurality of sub-scene networks, each of the sub-scene networks having independent scene parameters;

Performing feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed;

Based on the initial input features and the scene embedding features, resources are recommended to the target object.

On the other hand, an embodiment of the present application provides a data processing device, the device comprising:

An acquisition unit, configured to acquire initial input features of the data to be processed, wherein the initial input features include object features of a target object, resource features of a plurality of resources to be recommended, and scene features, wherein the scene features include scene identifiers;

A compression unit, used for performing feature compression processing on the initial input features to obtain low-dimensional input features;

A selection unit, configured to select, according to the scene identifier in the scene feature, a target sub-scene network matching the scene identifier from a plurality of sub-scene networks, each of the sub-scene networks having an independent scene parameter;

A processing unit, configured to perform feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed;

A recommendation unit is used to recommend resources to the target object based on the initial input features and the scene embedding features.

On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the data processing method described in any of the above embodiments.

On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the data processing method described in any of the above embodiments by calling the computer program stored in the memory.

On the other hand, an embodiment of the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the data processing method described in any of the above embodiments.

The embodiment of the present application obtains the initial input features of the data to be processed, wherein the initial input features include the object features of the target object, the resource features of multiple resources to be recommended, and the scene features, wherein the scene features include scene identifiers; the initial input features are subjected to feature compression processing to obtain low-dimensional input features; according to the scene identifier in the scene features, a target sub-scene network matching the scene identifier is selected from multiple sub-scene networks, and each sub-scene network has independent scene parameters; based on the target sub-scene network, the low-dimensional input features are subjected to feature processing to obtain the scene embedding features of the data to be processed; based on the initial input features and the scene embedding features, resources are recommended to the target object. The embodiment of the present application converts the initial input features into low-dimensional input features through feature compression processing, thereby reducing the computational complexity and improving the overall computational efficiency. According to the scene identifier in the scene features, a matching target sub-scene network is selected from multiple sub-scene networks, and each network has independent scene parameters, so that the recommendation system can be optimized for different scenarios and improve the scene adaptability. The network selection strategy driven by the scene identifier ensures that the model can dynamically adapt to the diverse scene requirements and improves the flexibility and pertinence of the recommendation system. Combining the initial input features and scene embedding features for resource recommendation ensures that the recommendation results not only consider the attributes of the target objects and resources, but also the scene context, thereby improving the accuracy and personalization level of the recommendation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a data processing method provided in an embodiment of the present application.

FIG. 2 is a schematic diagram of an application scenario of the data processing method provided in an embodiment of the present application.

FIG3 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application.

FIG4 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application.

DETAILED DESCRIPTION

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

The embodiments of the present application provide a data processing method, apparatus, storage medium, device and program product. Exemplarily, the data processing method of the embodiments of the present application can be executed by a computer device, wherein the computer device can be a terminal or a server. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, a smart speaker, a wearable smart device, a personal computer (PC), a smart car terminal and other devices, and the terminal can also include a client, which can be a video client, a shopping application client, a reading application client, a browser client or an instant messaging client. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

The embodiments of the present application can be applied to application scenarios such as artificial intelligence, machine learning, data processing, and resource recommendation.

The solutions provided in the embodiments of the present application involve technologies such as artificial intelligence data processing, which are specifically described by the following embodiments. The following are described in detail. It should be noted that the order of description of the following embodiments is not intended to limit the priority order of the embodiments.

An embodiment of the present application provides a data processing method, which can be executed by a terminal or a server, or by a terminal and a server together. The embodiment of the present application is described by taking the data processing method executed by a server as an example.

Please refer to Figures 1 and 2, Figure 1 is a flow chart of the data processing method provided in the embodiment of the present application, and Figure 2 is a schematic diagram of the application scenario of the data processing method provided in the embodiment of the present application. The method may include the following steps 110 to 150:

Step 110, obtaining initial input features of the data to be processed, wherein the initial input features include object features of the target object, resource features of a plurality of resources to be recommended, and scene features, wherein the scene features include scene identifiers.

In some embodiments, the obtaining of initial input features of the data to be processed includes:

Acquire data to be processed, wherein the data to be processed includes object association information of a target object, resource association information of a plurality of resources to be recommended, and scene information;

Feature extraction processing is performed on the object association information, the resource association information and the scene information to obtain the object feature of the target object, the resource feature of each of the resources to be recommended and the scene feature corresponding to each of the resources to be recommended.

In this step, we first need to obtain the data to be processed from various sources. This data includes the object association information of the target object and the resource association information of multiple resources to be recommended. This association information can be obtained in various ways, such as from database query, user input or other external systems. This data usually includes the object association information of the target object, the resource association information of multiple resources to be recommended, and scene information.

For example, the object association information of the target object involves detailed information about the user or the target object, including but not limited to: user identification ID (a string or number that uniquely identifies the user), user portrait information (such as gender, age, education level), geographic location (city, country, time zone), user historical behavior data (such as browsing history, click-through rate, playback, purchase history, rating history, collection history, etc.), and user preferences (interests and hobbies derived from analysis of the user's historical behavior data).

Resource association information of the resource to be recommended, which includes information about all resources that may be recommended to the target object, including but not limited to: resource ID (a string or number that uniquely identifies a resource), resource type (such as video, game, product, etc.), resource description (such as title, introduction, category, tag, etc.), resource attributes (such as price, size, color, brand, rating, number of comments, etc.), resource usage (such as sales ranking, popularity index, etc.).

Scenario information includes, but is not limited to: device type (such as mobile phones, tablets, computers, etc.), recommendation channels (such as social media, email, in-app recommendations, etc.), display methods (such as lists, grids, sliding cards, dynamic images, etc.), time and place (such as user active time, geographical location, etc.), and environmental factors (such as weather, holidays, etc.).

It is understandable that in the specific implementation of the present application, related data such as data to be processed is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Then, the data to be processed are subjected to feature extraction. This usually involves the use of various feature engineering techniques, such as text mining, image recognition, time series analysis, etc., to extract meaningful features from the raw data, which can effectively reduce the dimension of the data while retaining its main information. These features can include object features of the target object (such as user identification ID, age, gender, city, interest preferences, etc.), resource features of the resources to be recommended (such as resource ID, resource type, resource attributes, price, evaluation, etc.), and scene features (such as scene identification ID, timestamp, etc.).

The target object may be a user related to the target recommended resource. The resources to be recommended may include but are not limited to videos, games, movies, products, literary works, applications, etc.

For example, feature extraction is the process of converting the data to a format that can be used in a machine learning model. This process may include:

Text mining: Extract keywords and topics from user comments or resource descriptions in the data to be processed.

Image recognition: If the resource-associated information includes images or videos, use image recognition technology to extract visual features.

Time series analysis: Analyze the changing trends of user behavior in object association information over time.

Statistical analysis: Calculate statistical data corresponding to user historical behavior data, such as average rating, purchase frequency, click-through rate, number of plays, etc.

For example, before feature extraction, the original data to be processed usually needs to be cleaned and standardized to ensure data quality. For example, through noise removal, outliers and errors in the data are identified and removed. For example, through data filling, missing data is filled with appropriate methods. For example, through standardization, data is converted to a unified scale for easy comparison and processing.

For example, the extracted features are integrated together to form an initial input feature set. For example, all user-related features are integrated to form an object feature vector. For example, the relevant features of each resource to be recommended are integrated to form one or more resource feature vectors. For example, all scene-related features are integrated to form a scene feature vector.

Among them, scene features include scene identifiers (IDs), which are labels used to distinguish different recommendation scenes. Scene identifiers are the key to distinguishing different recommendation scenes, allowing the recommendation system to adjust its recommendation strategy according to different scenes. For example, if the scene identifier indicates that the user is using a mobile device, the recommendation system may prioritize resources suitable for mobile viewing.

Step 120: perform feature compression processing on the initial input features to obtain low-dimensional input features.

High-dimensional feature space will lead to a significant increase in computational complexity, especially in deep learning models, where a large number of features means more model parameters, which will take up more memory resources and computing time. Feature compression can effectively reduce the size of the model and speed up the training and reasoning process.

For example, the recommendation system shown in Figure 2 can compress the initial input features through the underlying compression network to map the high-dimensional initial input features to low-dimensional input features of lower dimensions, thereby reducing the subsequent inputs entering the sub-scenario network and reducing the computational cost. For example, the underlying compression network can be a deep neural network (DNN), and the DNN network can be a fully connected network, which compresses the initial input features by allowing the fully connected network to output a low-dimensional embed. The parameters of the fully connected network will participate in network training along with other network parameters, and no special design is required.

The underlying compression network can be a deep neural network (DNN), specifically, a fully connected network, which is used to compress the initial input features into low-dimensional embeddings. A fully connected network consists of multiple fully connected layers, each containing multiple neurons, and each neuron in a fully connected layer is connected to all neurons in the previous layer, which enables the network to learn the global features of the input data. The initial input features first pass through these fully connected layers, and each layer transforms and compresses the data. The weights and biases of the fully connected network are learnable parameters. During the training process, these parameters are updated through the back-propagation algorithm to minimize the difference between the predicted output and the true label. At the end of the network, a specific embedding layer is responsible for generating low-dimensional embeddings. The output of this embedding layer is the compressed feature representation. During the training process, a loss function needs to be defined to measure the difference between the model's prediction and the true label. The choice of loss function depends on the specific task of the recommendation system (such as click-through rate prediction, conversion rate prediction, etc.).

The low-dimensional input features obtained after compression will be used as input for subsequent steps, such as entering the scene-based network for further feature processing and resource recommendation. Since these features have lower dimensions, the cost of subsequent calculations can be reduced while retaining the key information in the original data.

In the recommendation system shown in Figure 2, the underlying compression network can first compress the features of each scene (such as user features, resource features, and scene features) separately, and then fuse the compressed low-dimensional features and send them to the subsequent scene-based network. This staged processing not only optimizes computational efficiency, but also provides a flexible basis for customized recommendations for each scene.

Step 130: According to the scene identifier in the scene feature, a target sub-scene network matching the scene identifier is selected from a plurality of sub-scene networks, each of the sub-scene networks having independent scene parameters.

In step 130, one or more sub-scene networks matching the target scene are selected from the pre-trained multiple sub-scene networks according to the scene identifier in the scene feature. These sub-scene networks are trained for different scenes or contexts, so they have different scene parameters and model structures. By selecting the sub-scene network that best matches the target scene, the accuracy and relevance of subsequent resource recommendations can be improved.

In some embodiments, the scene feature includes M scene identifiers, where M is a natural number greater than 0; and selecting a target sub-scene network matching the scene identifier from a plurality of sub-scene networks according to the scene identifier in the scene feature includes:

Each scene identifier in the M scene identifiers is matched with the preset scene identifier of each sub-scene network in the N sub-scene networks to obtain M target sub-scene networks matching the M scene identifiers, where N is greater than or equal to M.

As shown in Figure 2, the recommendation system establishes an independent sub-scenario network for each scene. Each sub-scenario network has independent scene parameters to prevent data from different scenes from interfering with each other. This means that each sub-scenario network can independently adjust its weights and biases to best adapt to the data pattern of its corresponding scene. This design enables each sub-scenario network to deeply understand the user needs and behavior characteristics in its scene. Each sub-scenario network can adopt a different model structure, such as a deep neural network (DNN), a convolutional neural network (CNN), or a recurrent neural network (RNN), depending on the characteristics of the scene data. For example, for visual content recommendations, CNN may be more applicable; while for sequence data, such as user behavior logs, RNN may be more effective.

The scene selector will automatically select the output of the corresponding sub-scene network as the final output based on the current scene identifier (ID).

Each business scenario has a corresponding scenario ID, which is input into the recommendation system shown in Figure 2 together with the training features. The scenario ID can also be considered as part of the training features. In the actual prediction stage, the scenario ID is also part of the initial input features.

Each sub-scenario network contains corresponding scene parameters, and the preset scene identifier corresponding to each sub-scenario network is defined before model training. The scene selector suppresses the output that does not belong to the current scene.

For example, the scene selector will compare the current scene ID with the preset current scene ID of each sub-scene network. If the two are the same, the output of the sub-scene network will remain unchanged. If the two are different, the output will be suppressed (set to 0). Finally, all the sub-scene outputs are added together to form the final scene embedding feature. In this way, the recommendation system can effectively concentrate computing resources to process the current scene while reducing the interference of irrelevant information.

The model structure composed of the underlying compression network, multiple sub-scene networks and scene selectors is a structure that can be embedded in other deep recommendation models. The output of this model structure (scene embedding features) can become the input of other deep recommendation models (the target recommendation model as shown in Figure 2), thereby enhancing the service capabilities of the corresponding model in different scenes.

In a multi-scenario recommendation system, each scenario (such as A recommendation channel, B recommendation channel, C recommendation channel, D recommendation channel, E short video unlimited stream, F short video internal stream, etc.) is regarded as a unique context environment with its own characteristics and user behavior patterns. In order to accurately capture these differences, the recommendation system builds multiple specialized sub-scenario networks, each of which specializes in understanding and predicting user preferences in the corresponding scenario. These sub-scenario networks match the preset scenario identifiers with the actual scenario identifiers to ensure that at any given moment, the recommendation system can call the target sub-scenario network that best suits the current context.

For example, the current scene is the F short video instream, corresponding to sub-scene network 3. In the prediction stage, each sub-scene network will make a prediction and output an embedding representation (embed). Assume that the outputs of each sub-scene network are embed1, embed2, ..., embedN-1, embedN. In the scene selector, except for sub-scene network 3, the embeds output by the remaining sub-scene networks will be multiplied by 0, and the final sum is embed3.

During the training phase, this scenario-based network structure adjusts the parameters of each network through back propagation, especially those networks that are frequently activated in the prediction phase, to continuously optimize their responsiveness to specific scenarios. For inapplicable scenarios, that is, networks whose outputs are masked, since their gradients are 0, they will not affect parameter updates, thus ensuring efficient learning and adaptability of the model.

For new or low-frequency scenarios, the system can adopt strategies to classify them into existing similar scenario categories, reducing network complexity while ensuring a certain degree of personalized recommendation quality. This approach reflects the flexibility and scalability of the system, so that even emerging or niche scenario needs can be met to a certain extent.

In addition, the sub-scenario network structure is designed as a module that can be seamlessly embedded into a larger recommendation system framework. This means that in addition to the explicit scenarios processed by the sub-scenario network, the entire recommendation system may also contain other components, such as basic feature extraction layer, user portrait modeling, content feature encoding, etc. These components work together to process scenarios or information that are not directly covered by the sub-scenario network, thereby achieving full-link personalized recommendation optimization.

In some embodiments, the method further comprises:

If there is no target sub-scenario network matching the scene identifier in the original multiple sub-scenario networks, a new sub-scenario network corresponding to the scene identifier is created.

In some embodiments, if there is no target sub-scenario network matching the scene identifier in the original multiple sub-scenario networks, creating a new sub-scenario network corresponding to the scene identifier includes:

If there is no target sub-scenario network matching the scenario identifier among the original multiple sub-scenario networks, detecting the data flow corresponding to the scenario identifier;

When the data traffic corresponding to the scenario identifier reaches a preset traffic threshold, a new scenario network corresponding to the scenario identifier is created;

When the creation time of the new sub-scenario network reaches a preset time threshold, the new sub-scenario network is added to the original multiple sub-scenario networks.

In some application scenarios, the recommendation system needs to be highly adaptable and scalable to cope with the ever-changing user needs and market environment. When the existing scenario-based network cannot cover all emerging scenarios, the system needs to have a mechanism to identify these new scenarios and create a matching scenario-based network in a timely manner.

For example, after receiving the scene identification, the system first checks whether there is a target scene network that fully matches it among the existing multiple scene networks. If it is found that there is no suitable network that can accurately serve the newly emerged scene, the system enters the next decision-making stage.

For example, to ensure efficient use of resources, the system does not immediately create a sub-scenario network for each new scenario. Instead, it starts monitoring the data traffic corresponding to the new scenario ID. This step is to verify whether the new scenario has sufficient user engagement and commercial value to be worth investing additional computing and storage resources. The data traffic can include the number of users corresponding to the scenario ID, request frequency, and interaction with other scenarios.

Among them, when the data traffic of a new scene reaches the preset traffic threshold, it indicates that the scene has attracted a certain amount of user attention and interaction. At this time, the system will consider it valuable to create a dedicated sub-scene network. The setting of the traffic threshold is a balancing strategy that aims to screen out scenes with real potential and demand, and avoid unnecessary resource allocation due to short-term or accidental traffic fluctuations. The preset traffic threshold is set in advance based on business needs and system capacity to determine whether a new scene has sufficient activity and influence, so that it is worth establishing and training a separate sub-scene network for it.

After deciding to create a new sub-scenario network, the system will use historical data or real-time data to initialize and train the new sub-scenario network based on the characteristics of the scenario, to ensure that the new sub-scenario network can quickly understand and predict user behavior in the new scenario.

The newly created sub-scenario network will not be added to the service immediately, but will have an "observation period". After a period of continuous training and optimization (preset time threshold, such as 5 days), when its performance is stable and the prediction effect is verified, the new sub-scenario network will be formally integrated into the original sub-scenario network system and participate in the actual recommendation service.

For example, if an e-commerce platform launches a new promotion, the promotion may bring about a new user interaction scenario. Through the above mechanism, the system can detect this new scenario, evaluate its traffic, and, if necessary, create a new scenario-based network to optimize the recommendation in this specific scenario.

The design of this series of steps not only ensures the flexibility of the recommendation system, enabling it to quickly adapt to the emergence of new scenarios, but also ensures the efficient use of resources by setting dual thresholds of traffic and time, avoiding over-investment in short-term or low-value scenarios. In addition, this mechanism also encourages the exploration and mining of new opportunities, which helps to improve the long-term competitiveness and user satisfaction of the overall recommendation system.

For example, the above embodiment is explained by taking a specific scenario as an example to intuitively understand how this process is applied in different recommendation channels and short video streaming scenarios.

Suppose there is a new content recommendation channel: A recommendation channel, which focuses on technology news. At present, there are multiple mature recommendation channel networks such as B, C, and D in the recommendation system, but there is no sub-scenario network specifically for technology news. First, the system monitors the user data traffic of A recommendation channel, including indicators such as page views, click-through rate, and user stay time. If the initial traffic is low, the system may not immediately create a dedicated sub-scenario network for it. Over time, if the user engagement of A recommendation channel gradually increases, its data traffic reaches a pre-set traffic threshold (such as 1 million views per day), which indicates that the channel has enough user activities to support independent model training. After confirming that A recommendation channel meets the traffic conditions, the system automatically triggers the creation of a new sub-scenario network, which will be initialized and trained for the characteristics of technology news and user preferences. In order to ensure the stability and accuracy of the new sub-scenario network, the system sets a preset time threshold (such as after a week of continuous training and testing) to continuously optimize the model performance during this period. After the new sub-scene network has been fully trained and its effectiveness has been verified, it will eventually be added to the existing multiple sub-scene networks and become a formal member, responsible for outputting the scene embedding features of recommendation channel A for application to personalized content recommendations under recommendation channel A.

The same logic can be applied to other scenarios such as B recommendation channel (such as focusing on entertainment news), C recommendation channel (such as focusing on financial information), D recommendation channel (such as focusing on live sports), E short video unlimited flow (such as exploration video recommendation), F short video internal flow (such as content flow of users following authors), etc. After reaching a specific traffic threshold and undergoing a certain period of training and verification, each new scenario or channel will generate a new sub-scenario network and integrate it into the sub-scenario network set to achieve more accurate content push.

In some embodiments, the creating a new sub-scenario network corresponding to the scenario identifier includes:

When there is a first preset scene identifier in the original multiple scene identifiers whose similarity with the scene identifier reaches a similarity threshold, migrating some parameters from the first scene identifier corresponding to the first preset scene identifier as initialization scene parameters of the new scene identifier;

According to the initial input features corresponding to the scene identifier, fine-tuning the initialization scene parameters of the new scene classification network to obtain the scene parameters corresponding to the scene identifier;

Based on the scene parameters corresponding to the scene identifier, a new sub-scene network corresponding to the scene identifier is generated.

In actual applications, the emergence of new scenarios is often not completely independent, and it may be similar to some existing scenarios. Therefore, when creating a new sub-scenario network, first check whether there are similar scenarios and consider using the parameters of these similar scenarios for migration, which can greatly improve the efficiency and performance of network construction.

First, the system compares the similarity between the new scene identifier and the scene identifiers in multiple existing scene networks. This is usually done by calculating some distance or similarity measure (such as cosine similarity) between the identifiers.

When the calculated similarity reaches a preset similarity threshold (eg, 80%), the system considers that an existing scene similar to the new scene is found, ie, the first preset scene identifier.

Then, the system will migrate some parameters from the first sub-scenario network corresponding to the first preset scenario identifier. These parameters may include network weights, bias terms, structural configurations, etc. The purpose of migration is to use the existing learning results as a strong starting point for the new network, reducing the time and data requirements for training from scratch.

Although the parameters of similar scenes are used as a basis, each scene is unique, so the next step is to fine-tune the migrated parameters. This step adjusts the initial scene parameters through optimization algorithms such as back propagation based on the initial input features corresponding to the new scene identifier. The fine-tuning process aims to make the model better adapt to the user preferences and behavior patterns unique to the new scene.

Then, based on the fine-tuned scene parameters, the system constructs a new scene-by-scene network that fully matches the scene identifier. This network not only inherits the general knowledge of similar scenes, but also incorporates personalized understanding of specific scenes, so that it can provide users with customized recommended content or services more accurately.

Then, the system will save the new sub-scenario network to an appropriate storage medium (such as a hard disk, database, etc.) for subsequent call and use. At the same time, the system will also update the relevant index and configuration information to ensure that the new sub-scenario network can be correctly identified and used.

For example, suppose that a personalized sub-scenario network is being designed for a new recommendation channel A corresponding to technology news and applied to the recommendation system. The existing recommendation system already contains several mature channel networks, such as recommendation channel B (for example, focusing on entertainment news) and recommendation channel C (for example, focusing on financial information). Although the content theme of recommendation channel A is different from that of existing channels, it has certain similarities with recommendation channel B in terms of user behavior patterns and certain content attributes. The similarity of the scene identifiers of the two reaches the preset similarity threshold after calculation. The system first analyzes the scene identifier of recommendation channel A and finds that it has a high similarity with the scene identifier of recommendation channel B, which meets the migration conditions. Since recommendation channel B already has a set of mature and well-performing scene parameters, the system decides to migrate some parameters from the sub-scenario network of recommendation channel B as the initialization parameters of the new sub-scenario network of recommendation channel A. Considering the differences between technology news and entertainment news, the system needs to fine-tune the migrated parameters according to the initial input features unique to recommendation channel A (such as user preference for technology keywords, timeliness of technology news, etc.). The fine-tuning process may involve training the model with a small amount of initial data to adapt to user behavior and content characteristics in the new scenario. The fine-tuned scene parameters are more in line with the needs of the A recommendation channel. At this point, the system generates a new scene-based network for the A recommendation channel based on these adjusted parameters. This network not only inherits some of the high-efficiency characteristics of the scene-based network of the B recommendation channel, but also incorporates customized optimization for science and technology news recommendations, so that the target recommendation model can more accurately recommend science and technology-related content to users.

Through this parameter migration and fine-tuning strategy based on similar scenarios, the new channel "A" can establish an efficient and customized recommendation service more quickly, reducing the time and resources required to build a model from scratch, while ensuring the accuracy of recommendations and user experience. Similar methods can also be applied to scenarios such as E short video infinite streaming and F short video internal streaming, and find suitable preset scenarios for parameter migration and optimization based on their respective characteristics.

In some embodiments, the creating a new sub-scenario network corresponding to the scenario identifier includes:

When there is no first preset scene identifier whose similarity with the scene identifier reaches a similarity threshold in the original multiple sub-scene networks, obtaining a randomly initialized scene parameter of the new sub-scene network;

According to the initial input features corresponding to the scene identifier, fine-tuning the randomly initialized scene parameters of the new scene segmentation network to obtain the scene parameters corresponding to the scene identifier;

Based on the scene parameters corresponding to the scene identifier, a new sub-scene network corresponding to the scene identifier is generated.

For example, this process can be specifically illustrated based on the E short video infinite streaming scenario, because this scenario is more dynamic and diverse than the traditional recommendation channel, and is very suitable for demonstrating how to create a brand new scenario-based network from scratch.

For example, we need to create a new scene-based network for a short video infinite stream focusing on "outdoor adventure" content for use in a personalized recommendation system. This field is relatively unique in the existing short video classification, and there is no preset scene identifier with a similarity threshold (for example, there may be comedy, food, fashion, etc.), so it is impossible to directly migrate parameters from the existing scene-based network.

Since there are no similar scenarios to draw on, the system will first randomly initialize the parameters for the new scenario network "outdoor adventure". Next, the system uses the initial input features collected for the specific scenario of "outdoor adventure" to fine-tune these randomly initialized parameters. These features may include the user's dwell time, likes, comments, sharing behaviors when browsing adventure-related videos, and the labels of the video itself (such as mountain climbing, diving adventures, wilderness survival skills, etc.). Through machine learning algorithms, the system gradually adjusts the parameters to better reflect the user's preferences and interaction habits for this type of content.

After fine-tuning, the system generates a new scene-based network for the infinite stream of "outdoor adventure" short videos based on the obtained scene parameters. The new scene-based network now has the ability to initially understand and predict user behavior in this specific content area, and can provide users with a continuous, personalized and attractive adventure video stream.

After the new scenario network is launched, the system will continue to collect feedback data, such as click-through rate, viewing time, conversion rate, etc., and further iterate and optimize model parameters to ensure more accurate recommendations and improve user experience and engagement. As user data accumulates, the recommendation network for the "outdoor adventure" scenario will gradually mature, and eventually form a highly customized and efficient recommendation system to meet users' exploration needs for outdoor adventure content.

In some embodiments, the scene feature includes a scene identifier and a scene level corresponding to the scene identifier;

The step of selecting a target sub-scenario network matching the scene identifier from a plurality of sub-scenario networks according to the scene identifier in the scene feature includes:

According to the scene identifier and the scene level, a target sub-scene network matching the scene identifier and the scene level is selected from a plurality of sub-scene networks.

For example, the scene feature includes not only the scene identifier, but also the scene level corresponding to the scene identifier. The scene level provides an indicator to measure the complexity or importance of the scene, which helps to more accurately match and process different scenes. Therefore, the process of selecting the target sub-scene network in step 130 can be performed based on the scene identifier and the scene level to achieve more detailed and accurate scene matching.

For example, a scenario-level mapping table is constructed, which records each scenario identifier and its corresponding scenario level. As a newly added consideration dimension, the scenario level reflects the complexity or importance of the scenario, and its value range and classification criteria can be customized according to specific business needs.

Then, we can also maintain a hierarchical scenario-based network architecture library, where scenario-based networks of different complexity and resource consumption correspond to different scenario levels. For example, low-level scenarios may correspond to lightweight scenario-based networks that focus on fast recommendations, while high-level scenarios match deep and complex scenario-based networks that pursue highly personalized and accurate recommendations.

After receiving the low-dimensional input features containing the scene identifier and the scene level, the scene-level mapping table is first queried to confirm the specific level of the current scene. Then, based on the scene identifier and the confirmed scene level, the most suitable target sub-scene network is searched in the hierarchical sub-scene network architecture library. During the matching process, the system will give priority to those sub-scene network models that meet both the scene identifier and the scene level requirements.

For example, in some cases, if resources are limited (for example, the system is under high load), the system may downgrade and select a sub-scenario network that is slightly lower than the current scenario level but still works effectively as the target sub-scenario network to ensure the stability and timeliness of the recommendation service.

The selection process of the target scenario network is not static, but can be dynamically adjusted based on multiple factors such as real-time system resource status, online performance feedback of each network model in the recommendation system, and user satisfaction. This means that for the same scenario identifier but at different time points or in different environments, the system may select different levels of scenario networks to achieve the best effect.

The following is an application scenario of personalized product recommendation on an e-commerce platform to illustrate this solution.

For example, there is an e-commerce platform where when users browse products, the system needs to provide personalized product recommendations based on the user's behavior and current environment. The platform defines a variety of scene identifiers, such as "daily browsing", "holiday promotions", "night shopping", etc., and sets different levels for each scene identifier to reflect the importance and complexity of the scene.

For example, user a opens an e-commerce platform at 10 pm and prepares to buy some household items. At this time, the scene is identified as "night shopping", and the system determines that the level of this scene is "intermediate" based on the user's historical behavior and time point. The system first processes the object features, product features, and scene features to obtain low-dimensional input features. Then the system consults the scene-level mapping table and confirms that the level of the "night shopping" scene is "intermediate". Then, based on the "night shopping" identification and the "intermediate" level, the system searches for a matching target sub-scene network in the hierarchical sub-scene network architecture library. For the "intermediate" scene, a target sub-scene network of medium complexity should be selected. The target recommendation model can generate a recommendation list based on the initial input features and the scene embedding features output by the target sub-scene network, comprehensively analyzing the user's past shopping habits, time-sensitive discount information, and user preferences for home products.

But suppose the system is facing high load and resource constraints at this time. To ensure service stability, the system decides to dynamically adjust and downgrade from the "intermediate" sub-scenario network corresponding to the "night shopping" logo to a "primary" sub-scenario network corresponding to the "night shopping" logo. The primary sub-scenario network may not rely so much on complex user behavior pattern analysis, and is more based on the user's basic preferences and recent popular products to generate scene embedding features. Although the degree of personalization is reduced, it can respond quickly to ensure that users can eventually get the recommendation list and maintain the smoothness of the service.

Next, the system will continue to monitor resource conditions, the online performance of each network model in the recommendation system (such as recommendation click-through rate, conversion rate), and user feedback. If the resource situation improves or it is found that the application of the primary sub-scenario network in the recommendation system has caused a decrease in user satisfaction, the system will switch back to a higher-level sub-scenario network in a timely manner. This dynamic adjustment mechanism means that even at different time points or when resource conditions change, for the same "night shopping" scenario, the system can select the most appropriate recommendation strategy based on actual conditions to ensure the quality and efficiency of the recommendation service.

In step 130, not only can the scenario-based networks be matched based on the scenario identifiers, but also a more refined and resource-sensitive network selection strategy can be implemented by integrating the scenario-level information, further improving the flexibility, efficiency and personalization level of the recommendation system. This helps to provide the most appropriate recommendation service in scenarios of various complexity and importance.

Step 140: perform feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed.

As shown in Figure 2, after the target sub-scene network is selected, the low-dimensional input features are input into the target sub-scene network for further feature processing. This process may include forward propagation of multi-layer neural networks, application of attention mechanisms, etc. to generate higher-level feature representations, namely scene embedding features. These features can better capture the behavior and preferences of the target object in a specific scene.

In some embodiments, the step of performing feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed includes:

Inputting the low-dimensional input features into the M target sub-scene networks respectively for feature processing to obtain sub-scene output features of each of the target sub-scene networks;

Aggregate the sub-scene output features of each of the target sub-scene networks to obtain the scene embedding features of the data to be processed.

In step 140, the system inputs the low-dimensional input features into M target sub-scene networks respectively. Here, M represents the number of selected target sub-scene networks, which are selected based on the current scene identifier (or the current scene identifier and scene level), and each network is optimized for a specific scene.

For example, in each target scene-by-scene network, low-dimensional input features go through a series of neural network layers, including convolutional layers, recurrent layers, fully connected layers, and possible advanced feature processing techniques such as attention mechanisms. These processes are designed to extract and learn higher-level representations of features, namely, scene-by-scene output features, which are more focused on capturing details and features related to specific scenes.

Then, the system aggregates the sub-scene output features from each target sub-scene network. For example, this aggregation can be achieved by a simple addition operation, that is, directly adding all sub-scene output features to obtain a comprehensive scene embedding feature. The scene embedding feature integrates information from different sub-scene networks, so it can more comprehensively reflect the characteristics of the data to be processed in different scenes.

Then, the comprehensive features obtained by aggregation are the scene embedding features of the data to be processed. This scene embedding feature not only contains the information of the original low-dimensional input features, but also incorporates in-depth learning and understanding of different scenarios, making its description of the data to be processed richer and more accurate.

Through this process, the system can select the most appropriate target sub-scene network for feature processing according to different scene identifiers (or different scene identifiers and scene levels), and generate a comprehensive scene embedding feature by aggregating the outputs of these target sub-scene networks. This method makes full use of the characteristics and advantages of different sub-scene networks, improves the expressiveness of scene embedding features, and thus helps to improve the accuracy and effectiveness of subsequent resource recommendations.

Step 150: Recommend resources to the target object based on the initial input features and the scene embedding features.

As shown in Figure 2, this step is to use the initial input features and scene embedding features to perform the resource recommendation task. This usually involves using a target recommendation model to predict the target object's prediction task for each resource to be recommended (such as click-through rate prediction, conversion rate prediction). Then, the resources are sorted or filtered according to the prediction results, and the target recommended resources that best meet the needs and preferences of the target object are selected as the output results of the target recommendation model for recommendation. This ensures that the recommendation is not only based on the match between the individual and the resource itself, but also fully considers the impact of the current scene, thereby improving the personalization and effectiveness of the recommendation.

In some embodiments, the recommending resources to the target object based on the initial input feature and the scene embedding feature includes:

Performing a prediction task based on the object feature of the target object, the resource features of multiple resources to be recommended, the scene feature and the scene embedding feature to obtain a task prediction value of the target object for each resource to be recommended, wherein the prediction task is one of a click rate prediction task and a conversion rate prediction task;

Determining a target recommended resource corresponding to the target object from the multiple resources to be recommended according to the task prediction value;

The target recommendation resource is recommended to the target object.

In step 150, firstly, a prediction task is performed based on the object features of the target object (such as user identification, user portrait features, historical behavior features, user preference features, etc.), resource features of multiple resources to be recommended (such as resource content features, resource types, resource tags, resource attributes, etc.), scene features (such as time, location, context information, device type, display method, etc.), and the scene embedding features generated in the previous steps. Prediction tasks usually include click-through rate prediction tasks, conversion rate prediction tasks, etc., which are used to estimate the interest or potential value of the target object in a specific resource.

The execution of the prediction task usually relies on a trained target recommendation model. The target recommendation model can capture the complex relationships between users, resources, and scenarios, and predict the user's potential interest in resources based on these relationships. In the prediction process, the initial input features and scene embedding features are used as inputs to the model. After a series of calculations within the model, the model finally outputs the user's predicted value for each resource to be recommended.

After obtaining the predicted values, the system will determine the target recommended resources corresponding to the target object from multiple resources to be recommended based on these predicted values. This usually involves operations such as sorting, filtering, or combining predicted values to ensure that the resources recommended to users not only meet their personal interests and preferences, but also meet the needs of the current scenario. For example, the system can sort resources according to the predicted values, and then select the top-ranked resources as the recommendation results. Alternatively, the system can filter the predicted values according to a preset threshold and only select resources with predicted values exceeding the threshold for recommendation. In addition, the system can also recommend resources in combination according to specific strategies to meet the diverse needs of users.

After determining the target recommended resources, the system will recommend these resources to the target object in an appropriate manner. The recommendation methods can be diversified, such as displaying recommended resources to users through email, SMS, in-app notifications, personalized page display, etc. At the same time, the system can also continuously optimize the recommendation results based on user feedback and behavior data to improve the accuracy and effectiveness of the recommendation.

In step 150, the scene embedding feature plays a crucial role. It integrates the behavior and preference information of the target object in a specific scene, making the recommendation result more in line with the needs of the current scene. By using the scene embedding feature, the system can more accurately capture the changes in user interests in different scenes, thereby providing more personalized recommendation services.

Step 150 realizes personalized resource recommendation for the target object by combining the initial input features and the scene embedding features. This step not only improves the accuracy and effect of the recommendation, but also fully considers the influence of the current scene, so that the recommendation results are more in line with the actual needs and preferences of the user. In practical applications, the performance and user experience of the resource recommendation system can be further improved by continuously optimizing and improving the recommendation algorithm and model.

For example, in a recommendation system, when faced with multi-scenario applications, each scenario may have its own unique recommendation requirements and user behavior patterns. The following will illustrate the above steps 110 to 150 by combining scenarios such as A recommendation channel, B recommendation channel, C recommendation channel, D recommendation channel, E short video unlimited stream and F short video internal stream:

In step 110, the system obtains initial input features related to the target object (user) and the resources to be recommended. These initial input features include the user's object features (such as age, gender, historical behavior, etc.), resource features of multiple resources to be recommended (such as resource type, label, rating, etc.), and scene features (such as scene identification such as A recommended channel, B recommended channel, etc.).

In step 120, in order to reduce computational complexity and improve model efficiency, the system may compress the initial input features through the underlying compression network as shown in FIG. 2 to map the high-dimensional initial input features to lower-dimensional low-dimensional input features.

In step 130, the system selects a target sub-scenario network that matches the current scene identifier from a plurality of pre-trained sub-scenario networks according to the scene identifier (such as A, B, C, D, E, F, etc.) in the scene feature. Each sub-scenario network is optimized for a specific scene and has independent scene parameters to capture the user behavior patterns and resource characteristics in the scene.

In step 140, the low-dimensional input features are input into the selected target sub-scene network, and the scene embedding features of the data to be processed are obtained through multi-layer processing and transformation of the network. The scene embedding features contain the potential relationship and feature representation between the user and the resource to be recommended in the current scene.

In step 150, the system combines the initial input features and the scene embedding features and uses the target recommendation model to make resource recommendations.

For example, for the scenario of channel A recommendation, the system may focus more on recommending resources that are highly relevant to the user's historical behavior and interests. Therefore, when predicting the model, more consideration will be given to the user's object features and the long-term interest part in the scene embedding features.

For example, in the scenario of B recommending channel, the channel may focus more on recommending popular or newly launched resources. Therefore, when predicting the model, more consideration will be given to the popularity and freshness of resource features and scene embedding features.

For example, in the scenario of C recommending a channel, the channel may be a recommendation channel for a specific field or theme, such as technology, entertainment, etc. In this scenario, the system will make recommendations based on the user's field preferences and the field relevance in the scene embedding features.

For example, in the scenario of D recommending channel, the channel may be a highly personalized channel, and the system will make dynamic recommendations based on the user's real-time behavior and scenario embedding features.

For example, in the scenario of infinite streaming of E short videos, the system needs to recommend short videos to users quickly and continuously. Therefore, an efficient model structure will be used, combined with the user's immediate interests in the scene embedding features for recommendation.

For example, for the F short video in-stream scenario, when a user is watching a short video, the system will recommend related short videos based on the content of the video and the user's viewing behavior, as well as the contextual information in the scene embedded features.

During the recommendation process, the system will sort or filter resources based on the results of prediction tasks (such as click-through rate prediction, conversion rate prediction, etc.), and ultimately select the target recommended resources that best meet user needs and preferences for recommendation. At the same time, the system will continue to optimize the recommendation results based on user feedback and behavior data to improve the accuracy and effectiveness of the recommendation.

All of the above technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.

The embodiment of the present application obtains the initial input features of the data to be processed, wherein the initial input features include the object features of the target object, the resource features of multiple resources to be recommended, and the scene features, wherein the scene features include scene identifiers; the initial input features are subjected to feature compression processing to obtain low-dimensional input features; according to the scene identifier in the scene features, a target sub-scene network matching the scene identifier is selected from multiple sub-scene networks, and each sub-scene network has independent scene parameters; based on the target sub-scene network, the low-dimensional input features are subjected to feature processing to obtain the scene embedding features of the data to be processed; based on the initial input features and the scene embedding features, resources are recommended to the target object. The embodiment of the present application converts the initial input features into low-dimensional input features through feature compression processing, thereby reducing the computational complexity and improving the overall computational efficiency. According to the scene identifier in the scene features, a matching target sub-scene network is selected from multiple sub-scene networks, and each network has independent scene parameters, so that the recommendation system can be optimized for different scenarios and improve the scene adaptability. The network selection strategy driven by the scene identifier ensures that the model can dynamically adapt to the diverse scene requirements and improves the flexibility and pertinence of the recommendation system. Combining the initial input features and scene embedding features for resource recommendation ensures that the recommendation results not only consider the attributes of the target objects and resources, but also the scene context, thereby improving the accuracy and personalization level of the recommendation.

In order to better implement the data processing method of the embodiment of the present application, the embodiment of the present application also provides a data processing device. Please refer to Figure 3, which is a schematic diagram of the structure of the data processing device provided by the embodiment of the present application. The data processing device 200 may include:

An acquisition unit 210 is used to acquire initial input features of the data to be processed, wherein the initial input features include object features of a target object, resource features of a plurality of resources to be recommended, and scene features, wherein the scene features include scene identifiers;

A compression unit 220, configured to perform feature compression processing on the initial input features to obtain low-dimensional input features;

A selection unit 230, configured to select a target sub-scenario network matching the scene identifier from a plurality of sub-scenario networks according to the scene identifier in the scene feature, each of the sub-scenario networks having independent scene parameters;

A processing unit 240 is used to perform feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed;

The recommendation unit 250 is used to recommend resources to the target object based on the initial input features and the scene embedding features.

In some embodiments, the scene features include M scene identifiers, where M is a natural number greater than 0; the selection unit 230 can be used to: match each of the M scene identifiers with a preset scene identifier of each sub-scene network in N sub-scene networks, respectively, to obtain M target sub-scene networks that match the M scene identifiers, where N is greater than or equal to M.

In some embodiments, the processing unit 240 can be used to input the low-dimensional input features into the M target sub-scene networks for feature processing respectively to obtain the sub-scene output features of each of the target sub-scene networks; aggregate the sub-scene output features of each of the target sub-scene networks to obtain the scene embedding features of the data to be processed.

In some embodiments, the data processing device 200 further includes:

A creating unit is used to create a new sub-scenario network corresponding to the scene identifier if there is no target sub-scenario network matching the scene identifier in the original multiple sub-scenario networks.

In some embodiments, the creation unit can be used to: if there is no target sub-scenario network matching the scene identifier in the original multiple sub-scenario networks, then detect the data traffic corresponding to the scene identifier; when the data traffic corresponding to the scene identifier reaches a preset traffic threshold, create a new sub-scenario network corresponding to the scene identifier; when the creation time of the new sub-scenario network reaches a preset time threshold, add the new sub-scenario network to the original multiple sub-scenario networks.

In some embodiments, when creating a new sub-scenario network corresponding to the scene identifier, the creation unit may be used to: when there is a first preset scene identifier in the original multiple sub-scenario networks whose similarity with the scene identifier reaches a similarity threshold, migrate some parameters from the first sub-scenario network corresponding to the first preset scene identifier as initialization scene parameters of the new sub-scenario network;

According to the initial input features corresponding to the scene identifier, the initialization scene parameters of the new sub-scene network are fine-tuned to obtain the scene parameters corresponding to the scene identifier; based on the scene parameters corresponding to the scene identifier, a new sub-scene network corresponding to the scene identifier is generated.

In some embodiments, when creating a new sub-scene network corresponding to the scene identifier, the creation unit can be used to: when there is no first preset scene identifier in the original multiple sub-scene networks whose similarity with the scene identifier reaches a similarity threshold, obtain the randomly initialized scene parameters of the new sub-scene network; according to the initial input features corresponding to the scene identifier, fine-tune the randomly initialized scene parameters of the new sub-scene network to obtain the scene parameters corresponding to the scene identifier; based on the scene parameters corresponding to the scene identifier, generate a new sub-scene network corresponding to the scene identifier.

In some embodiments, the scene features include a scene identifier and a scene level corresponding to the scene identifier; the selection unit 230 can be used to: select a target sub-scene network that matches the scene identifier and the scene level from multiple sub-scene networks based on the scene identifier and the scene level.

In some embodiments, the recommendation unit 250 can be used to: perform a prediction task based on the object characteristics of the target object, the resource characteristics of multiple resources to be recommended, the scene characteristics and the scene embedded characteristics to obtain a task prediction value of the target object for each of the resources to be recommended, wherein the prediction task is one of a click-through rate prediction task and a conversion rate prediction task; determine the target recommended resource corresponding to the target object from the multiple resources to be recommended according to the task prediction value; and recommend the target recommended resource to the target object.

In some embodiments, the acquisition unit 210 can be used to: acquire data to be processed, the data to be processed including object association information of a target object, resource association information of multiple resources to be recommended, and scene information; perform feature extraction processing on the object association information, the resource association information and the scene information to obtain the object features of the target object, the resource features of each of the resources to be recommended and the scene features corresponding to each of the resources to be recommended.

It should be noted that the functions of each module in the data processing device 200 in the embodiment of the present application can correspond to the specific implementation method of any embodiment in the above-mentioned method embodiments, and will not be repeated here.

Each unit in the above device can be implemented in whole or in part by software, hardware or a combination thereof. Each unit can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each unit.

For example, the data processing device 200 may be integrated in a terminal or a server that has a storage device and a processor installed therein and has computing capabilities, or the data processing device 200 may be the terminal or the server.

In some embodiments, the present application further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

FIG4 is a schematic structural diagram of a computer device provided in an embodiment of the present application. As shown in FIG4 , a computer device 300 may include: a communication interface 301, a memory 302, a processor 303, and a communication bus 304. The communication interface 301, the memory 302, and the processor 303 communicate with each other through the communication bus 304. The communication interface 301 is used for the device 300 to communicate data with an external device. The memory 302 can be used to store software programs and modules, and the processor 303 runs the software programs and modules stored in the memory 302, such as the software programs of the corresponding operations in the aforementioned method embodiment.

In some embodiments, the processor 303 may call software programs and modules stored in the memory 302 to perform the following operations:

Acquire initial input features of the data to be processed, wherein the initial input features include object features of a target object, resource features of multiple resources to be recommended, and scene features, wherein the scene features include scene identifiers; perform feature compression processing on the initial input features to obtain low-dimensional input features; based on the scene identifier in the scene features, select a target sub-scene network that matches the scene identifier from multiple sub-scene networks, each of the sub-scene networks having independent scene parameters; perform feature processing on the low-dimensional input features based on the target sub-scene network to obtain scene embedding features of the data to be processed; and recommend resources to the target object based on the initial input features and the scene embedding features.

In some embodiments, the computer device 300 may be integrated in a terminal or server having a storage device and a processor installed therein and having computing capabilities, or the computer device 300 may be the terminal or server. The terminal may be a smart phone, a tablet computer, a laptop computer, a smart TV, a smart speaker, a wearable smart device, a personal computer, or other devices. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

The present application also provides a computer-readable storage medium for storing a computer program. The computer-readable storage medium can be applied to a computer device, and the computer program enables the computer device to execute the corresponding processes in the above-mentioned methods in the embodiments of the present application, which will not be described in detail for the sake of brevity.

The present application also provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the corresponding processes in the above-mentioned methods in the embodiments of the present application, which will not be described here for the sake of brevity.

The present application also provides a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the corresponding processes in the above-mentioned methods in the embodiments of the present application, which will not be described here for the sake of brevity.

It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer or a server) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (14)

1. A data processing method, characterized in that the method comprises: Acquire initial input features of the data to be processed, wherein the initial input features include object features of a target object, resource features of a plurality of resources to be recommended, and scene features, wherein the scene features include a scene identifier; Performing feature compression processing on the initial input features to obtain low-dimensional input features; According to the scene identifier in the scene feature, a target sub-scene network matching the scene identifier is selected from a plurality of sub-scene networks, each of the sub-scene networks having independent scene parameters; Performing feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed; Based on the initial input features and the scene embedding features, resources are recommended to the target object. 2. The data processing method according to claim 1, wherein the scene feature comprises M scene identifiers, where M is a natural number greater than 0; The step of selecting a target sub-scenario network matching the scene identifier from a plurality of sub-scenario networks according to the scene identifier in the scene feature includes: Each scene identifier in the M scene identifiers is matched with the preset scene identifier of each sub-scene network in the N sub-scene networks to obtain M target sub-scene networks matching the M scene identifiers, where N is greater than or equal to M. 3. The data processing method according to claim 2, characterized in that the step of performing feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed comprises: Inputting the low-dimensional input features into the M target sub-scene networks respectively for feature processing to obtain sub-scene output features of each of the target sub-scene networks; Aggregate the sub-scene output features of each of the target sub-scene networks to obtain the scene embedding features of the data to be processed. 4. The data processing method according to claim 1, characterized in that the method further comprises: If there is no target sub-scenario network matching the scene identifier in the original multiple sub-scenario networks, a new sub-scenario network corresponding to the scene identifier is created. 5. The data processing method according to claim 4, characterized in that if there is no target sub-scenario network matching the scene identifier in the original multiple sub-scenario networks, creating a new sub-scenario network corresponding to the scene identifier comprises: If there is no target sub-scenario network matching the scenario identifier among the original multiple sub-scenario networks, detecting the data flow corresponding to the scenario identifier; When the data traffic corresponding to the scenario identifier reaches a preset traffic threshold, a new scenario network corresponding to the scenario identifier is created; When the creation time of the new sub-scenario network reaches a preset time threshold, the new sub-scenario network is added to the original multiple sub-scenario networks. 6. The data processing method according to claim 5, wherein the step of creating a new sub-scenario network corresponding to the scenario identifier comprises: When there is a first preset scene identifier in the original multiple scene identifiers whose similarity with the scene identifier reaches a similarity threshold, migrating some parameters from the first scene identifier corresponding to the first preset scene identifier as initialization scene parameters of the new scene identifier; According to the initial input features corresponding to the scene identifier, fine-tuning the initialization scene parameters of the new scene classification network to obtain the scene parameters corresponding to the scene identifier; Based on the scene parameters corresponding to the scene identifier, a new sub-scene network corresponding to the scene identifier is generated. 7. The data processing method according to claim 5, wherein the step of creating a new sub-scenario network corresponding to the scenario identifier comprises: When there is no first preset scene identifier whose similarity with the scene identifier reaches a similarity threshold in the original multiple sub-scene networks, obtaining a randomly initialized scene parameter of the new sub-scene network; According to the initial input features corresponding to the scene identifier, fine-tuning the randomly initialized scene parameters of the new scene segmentation network to obtain the scene parameters corresponding to the scene identifier; Based on the scene parameters corresponding to the scene identifier, a new sub-scene network corresponding to the scene identifier is generated. 8. The data processing method according to claim 1, wherein the scene feature comprises a scene identifier and a scene level corresponding to the scene identifier; The step of selecting a target sub-scenario network matching the scene identifier from a plurality of sub-scenario networks according to the scene identifier in the scene feature includes: According to the scene identifier and the scene level, a target sub-scene network matching the scene identifier and the scene level is selected from a plurality of sub-scene networks. 9. The data processing method according to any one of claims 1 to 8, characterized in that the recommending resources to the target object based on the initial input features and the scene embedding features comprises: Performing a prediction task based on the object feature of the target object, the resource features of multiple resources to be recommended, the scene feature and the scene embedding feature to obtain a task prediction value of the target object for each resource to be recommended, wherein the prediction task is one of a click rate prediction task and a conversion rate prediction task; Determining a target recommended resource corresponding to the target object from the multiple resources to be recommended according to the task prediction value; The target recommendation resource is recommended to the target object. 10. The data processing method according to any one of claims 1 to 8, wherein the step of obtaining initial input features of the data to be processed comprises: Acquire data to be processed, wherein the data to be processed includes object association information of a target object, resource association information of a plurality of resources to be recommended, and scene information; Feature extraction processing is performed on the object association information, the resource association information and the scene information to obtain the object feature of the target object, the resource feature of each of the resources to be recommended and the scene feature corresponding to each of the resources to be recommended. 11. A data processing device, characterized in that the device comprises: An acquisition unit, configured to acquire initial input features of the data to be processed, wherein the initial input features include object features of a target object, resource features of a plurality of resources to be recommended, and scene features, wherein the scene features include scene identifiers; A compression unit, used for performing feature compression processing on the initial input features to obtain low-dimensional input features; A selection unit, configured to select, according to the scene identifier in the scene feature, a target sub-scene network matching the scene identifier from a plurality of sub-scene networks, each of the sub-scene networks having independent scene parameters; A processing unit, configured to perform feature processing on the low-dimensional input features based on the target scene-based network to obtain scene embedding features of the data to be processed; A recommendation unit is used to recommend resources to the target object based on the initial input features and the scene embedding features. 12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the data processing method according to any one of claims 1 to 10. 13. A computer device, characterized in that the computer device comprises a processor and a memory, the memory stores a computer program, and the processor executes the data processing method according to any one of claims 1 to 10 by calling the computer program stored in the memory. 14. A computer program product, comprising computer instructions, characterized in that when the computer instructions are executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.