CN120563214A - Store personalized recommendation interaction method and system combined with user portrait - Google Patents

Abstract

The present invention provides a personalized store recommendation interaction method and system combined with user portraits. First, interaction signal sequences such as user position movement signals, product contact signals and voice interaction signals in the store are obtained, and a user intention feature vector is generated through a spatiotemporal feature encoder. Then, a scene feature extractor is used to obtain the current interaction environment characteristics of the store. Then, a recommendation decision network is used to perform feature fusion reasoning on the user intention feature vector and the interaction environment characteristics to generate personalized recommendation instructions including content identification, trigger location and response method. Finally, the personalized recommendation instructions are executed and user response signals are collected to update the encoding parameters of the spatiotemporal feature encoder and the inference weights of the recommendation decision network. In this way, user needs and store environment characteristics can be accurately captured, personalized recommendations can be achieved, and customer shopping experience and store sales conversion rate can be improved.

Description

Store personalized recommendation interaction method and system combined with user portrait

Technical Field

The invention relates to the technical field of digital services, in particular to a store personalized recommendation interaction method and system combined with user portraits.

Background

In the retail industry, personalized recommendations of stores have important significance for improving the shopping experience of customers and increasing commodity sales conversion rate. Traditional store recommendation methods mainly depend on manual experience and simple commodity classification display, and personalized requirements of customers are difficult to accurately grasp.

With the continuous development of information technology, some stores start to introduce recommendation systems based on user history consumption records, but this approach has obvious limitations. On the one hand, the historical consumption record can only reflect the past purchasing behavior of the user, and the current real-time demand and interest change of the user at the store can not be captured in time. On the other hand, most of the existing recommendation systems only consider the information of the users, but neglect the current interactive environment characteristics of the store, such as spatial distribution of commodities, equipment states, time-sequence flow and the like, and these environment factors can have important influence on the purchase decision of the users. For example, certain merchandise placed in a particular area may attract more attention from customers, equipment failure or crowded traffic during peak hours may also affect the user's shopping experience and decisions.

Disclosure of Invention

In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a method for interaction of personalized store recommendation in combination with a user portrait, where the method includes:

acquiring a user interaction signal sequence, wherein the interaction signal sequence comprises a position movement signal, a commodity contact signal and a voice interaction signal of a user in a store;

Performing space-time feature coding processing on the interactive signal sequence through a space-time feature coder to generate a user intention feature vector, wherein the user intention feature vector comprises a requirement explicit feature, a requirement implicit feature and a requirement association feature;

Acquiring current interaction environment characteristics of a store by using a scene characteristic extractor, wherein the interaction environment characteristics comprise commodity space distribution characteristics, equipment state characteristics and time sequence flow characteristics;

performing feature fusion reasoning processing on the user intention feature vector and the interaction environment feature through a recommendation decision network to generate a personalized recommendation instruction, wherein the personalized recommendation instruction comprises a content identifier, a trigger position and a response mode;

executing the personalized recommendation instruction and collecting a user response signal, wherein the user response signal is used for updating the coding parameters of the space-time characteristic encoder and the inference weight of the recommendation decision network.

In yet another aspect, an embodiment of the present invention further provides a store personalized recommendation interaction system combined with a user portrait, including a processor, and a machine-readable storage medium, where the machine-readable storage medium is connected to the processor, and the machine-readable storage medium is used to store a program, an instruction, or a code, and the processor is used to execute the program, the instruction, or the code in the machine-readable storage medium, so as to implement the method described above.

Based on the above aspects, the embodiment of the invention obtains the interactive signal sequences such as the position moving signal, the commodity contact signal, the voice interactive signal and the like of the user in the store, and performs space-time feature coding processing on the interactive signal sequences by utilizing the space-time feature coder to generate the user intention feature vector comprising the requirement explicit feature, the requirement implicit feature and the requirement association feature, so that the real-time requirement and intention of the user in the store can be comprehensively and accurately captured. Meanwhile, the scene feature extractor is utilized to acquire the current interaction environment features of the store, including the commodity space distribution features, the equipment state features and the time sequence flow features, and the influence of the store environment on the purchase decision of the user is fully considered. And carrying out feature fusion reasoning processing on the user intention feature vector and the interaction environment feature through a recommendation decision network to generate a personalized recommendation instruction comprising a content identifier, a trigger position and a response mode, so that the recommendation content can be accurately matched and pushed according to the real-time requirements of the user and the store environment. The personalized recommendation instruction is executed, the user response signals are collected, the encoding parameters of the space-time feature encoder and the inference weight of the recommendation decision network are updated by utilizing the signals, the dynamic optimization and the self-adaptive adjustment of a recommendation system are realized, the recommendation effect can be continuously improved along with the changes of user behaviors and store environments, the accuracy and the effectiveness of personalized recommendation of stores can be remarkably improved, the shopping experience and the satisfaction of customers are improved, and the commodity sales conversion rate and the competitiveness of stores are further improved.

Drawings

FIG. 1 is a schematic diagram of an execution flow of a method for a store personalized recommendation interaction in combination with user portraits according to an embodiment of the present invention.

FIG. 2 is a schematic diagram of exemplary hardware and software components of a store personalized recommendation interaction system incorporating user portraits provided by an embodiment of the invention.

Detailed Description

The present invention is specifically described below with reference to the accompanying drawings, and fig. 1 is a schematic flow chart of a method for user portrait-combined store personalized recommendation interaction according to an embodiment of the present invention, and the method for user portrait-combined store personalized recommendation interaction is described in detail below.

Step S110, a user interaction signal sequence is obtained, wherein the interaction signal sequence comprises a position movement signal, a commodity contact signal and a voice interaction signal of a user in a store.

In the process of acquiring the user interaction signal sequence, various sensor devices can be used for respectively acquiring different types of signals. In the process of acquiring the user interaction signal sequence, the data authorization permission is a basis for ensuring legal compliance operation of the whole recommendation interaction system, and is a key link for protecting privacy and rights and interests of users.

Before data acquisition, the user can be clearly informed of the purpose, range, use mode, storage period and other information of the data acquisition. The special data acquisition explanation mark is arranged at the obvious positions of the store entrance, the commodity display rack side, the interaction area and the like, and the acquired position movement signal, commodity contact signal and voice interaction signal are elaborated to be used for personalized recommendation service so as to improve the shopping experience of a user in the store.

The user is guided to perform a data authorization operation when entering the store. The user can choose whether to agree to authorize the acquisition of data or not through the self-service terminal equipment arranged at the store entrance, clear and definite authorization options are provided, and the user can choose all the authorizations, part of the authorizations or refusal of the authorizations according to own wish. If the user selects partial authorization, data collection is performed in a range specified by the user. For example, the user may only agree to collect the position movement signal, and the system will not collect the merchandise contact signal and the voice interaction signal.

For data collection with consent to authorization, the minimum necessary principle is followed. When the position moving signal is acquired, only the corresponding data of the position coordinate point and the timestamp related to the personalized recommendation is recorded, and other irrelevant information is not excessively acquired. For commodity contact signals, only the contact time length, the contact force and the contact position data are collected, and other privacy information of a user cannot be related. When the voice interaction signals are collected, voice contents are strictly screened and filtered, only semantic information related to commodity recommendation is extracted, and sensitive information collection is avoided.

In order to protect privacy sensitive data of users, various privacy protection and anti-disclosure technical means can be adopted. In the data transmission process, the data is encrypted by adopting an encryption algorithm, so that the data is ensured not to be stolen or tampered in the transmission process. For example, the position movement signal, the merchandise contact signal, and the voice interaction signal are encrypted using a symmetric encryption algorithm, and only authorized system components can decrypt and process the data.

In terms of data storage, safe and reliable storage devices and storage architectures are employed. The data is stored in a special database and strict access rights control is set. Only authorized system administrators and related personnel can access and process this data, and access records can be recorded and audited in detail. Meanwhile, the stored data is backed up regularly to prevent the data from being lost or damaged.

In the data processing process, the privacy sensitive data is subjected to desensitization processing. And carrying out replacement or deletion operation on sensitive contents such as user identity information, contact information and the like in the voice interaction signal, and only retaining semantic information related to commodity recommendation. For example, the specific name mentioned by the user is replaced with "customer", the telephone number is replaced with "contact information", and the like.

During the data use process, related laws and regulations and user authorization ranges are strictly complied with. The privacy-sensitive data of the user is not used for other unauthorized purposes nor is the data sold or shared to a third party. If the data analysis or model training is needed to be carried out in cooperation with the third party, the user definitely authorization needs to be obtained in advance, and a strict data confidentiality protocol is signed with the third party, so that the safety and privacy of the data are ensured.

After executing the personalized recommendation instructions and collecting the user response signals, the user response signals also follow strict privacy protection and data authorization principles. The user response signal is used for updating the coding parameters of the space-time feature encoder and the inference weights of the recommendation decision network, but anonymizing the data is carried out in the using process, so that the specific user identity cannot be identified from the data.

Through the data authorization permission and privacy protection measures, the whole recommendation interaction system of the embodiment is legal and compliant in the data acquisition, processing and use processes, the privacy and rights of users are protected, and the situations of legal violation, fair violation, prejudice and the like are avoided. Meanwhile, the recommendation service which is safer, more reliable and more personalized is provided for the user.

In the subsequent system operation process, the change of laws and regulations and the feedback of users are continuously focused, and the data authorization permission and privacy protection measures are continuously optimized. And (3) carrying out security audit and vulnerability scanning on the system regularly, and timely finding and solving potential security problems. Meanwhile, training and education of staff are enhanced, legal consciousness and privacy protection consciousness of the staff are improved, and safe and stable operation of the whole system is ensured.

In addition, in order to make the user more aware of the usage of data and their own interests, data usage reports are regularly provided to the user. The data usage report specifies the type of data collected, the manner of use, the storage period, and the measures taken to protect the privacy of the user. The user can adjust the authorization setting of the user at any time according to the report content, so that the privacy of the user is ensured to be fully protected.

During interaction with the user, a dedicated feedback channel is provided. The user can feed back any problems or suggestions in the data acquisition and use process through modes such as a customer service desk, an online customer service system or a feedback mailbox in a store, timely process the feedback of the user, and correspondingly adjust and improve according to the requirements of the user.

In order to ensure the validity and legality of the data authorization permission, technical means such as digital signature, timestamp and the like can be adopted to record and verify the authorization information of the user. When a user performs an authorization operation, a unique digital signature is generated, and an authorization time stamp is recorded, and the information is stored in a safe database for subsequent inquiry and verification. In the data use process, the validity of the authorization information is verified periodically, so that the use of the data is ensured to be always in the user authorization range. Meanwhile, an audit mechanism for data use is established. And (3) comprehensively auditing and monitoring the data acquisition, processing, storage and use processes. The audit log can record information such as the operation process, the operator, the operation time and the like of the data in detail so as to trace back and find out reasons in time when problems occur. The audit results are analyzed and evaluated regularly to find potential risks and problems and timely take measures to improve.

In terms of data sharing and collaboration, related laws and regulations and user authorization are strictly adhered to. If data sharing with a third party partner is required, explicit authorization of the user is obtained in advance, and a strict data sharing protocol is signed with the partner. The protocol can clearly define the use range, confidentiality responsibility, security measures and other contents of the data, so that the partner can also comply with the requirements of privacy protection and data security. In the aspect of data destruction, when the data reaches the storage period or is no longer needed to be used, the data destruction is carried out according to a specified flow. And a safe and reliable destroying technology is adopted to ensure that the data cannot be recovered and utilized. The data destruction process may be recorded and audited to prove that the data has been safely destroyed.

Through the operation, better and personalized recommendation service can be provided for the user, the privacy and the rights and interests of the user are protected, and the trust and satisfaction of the user are improved. In the process of continuous development and perfection, the change of laws and regulations and the progress of technology are continuously focused, the privacy protection mechanism of the system is continuously optimized, and the recommended interaction system of the embodiment is ensured to always meet the latest safety and privacy requirements.

On the basis of the above, an embodiment of the sub-steps of the present step is described in detail below.

For example, in step S111, corresponding data of a position coordinate point and a time stamp triggered in the moving process of a user is collected through a sensor array arranged on the ground of a store, and a position moving signal is generated.

In a store scene, a sensor array consisting of a plurality of sensors is uniformly distributed on the ground. As the user moves within the store, the pressure exerted by the footsteps triggers the sensor at the corresponding location, thereby registering the relevant data.

For example, step S1111 obtains a trigger time stamp and a node number for each sensor node in the sensor array.

When the user steps into the store and starts to move, the sensor nodes are triggered in turn. And when each trigger is performed, recording the accurate time when the sensor node is triggered, namely, the time stamp, and simultaneously recording the unique number of the sensor node. For example, the user passes through a store area, and the sensor node A1 of the area is triggered, and the trigger time is recorded as T1 and the node number A1.

And step S1112, converting the node numbers into two-dimensional coordinate positions according to the coordinate mapping relation of the sensor node numbers and the store electronic map.

In the initial setting stage, a mapping relation between the sensor node numbers and the two-dimensional coordinates of the shop electronic map is established, the mapping relation can be embodied through a pre-constructed mapping table, and the mapping table stores two-dimensional coordinate information corresponding to each sensor node number. After the node number A1 is obtained, the corresponding two-dimensional coordinate positions (X1, Y1) of the sensor can be obtained by inquiring the mapping table, wherein X1 represents the abscissa and Y1 represents the ordinate, and the two coordinate values can accurately position the specific position of the sensor in a store.

And step S1113, storing the time stamp and the corresponding coordinate position in an associated mode, and generating corresponding data of the position coordinate point and the time stamp.

After the time stamp and the corresponding two-dimensional coordinate position of each sensor node are acquired, the sensor nodes can be associated and stored. Taking a key-value pair in the data structure as an example, a timestamp may be used as a key, and the corresponding two-dimensional coordinate position may be used as a value, to form a data set containing time and position information. For example, the time T1 is stored in association with the coordinates (X1, Y1), thereby obtaining corresponding data of a position coordinate point and a time stamp.

Step S1114, performing de-duplication processing on the corresponding data, performing linear interpolation processing on the de-duplicated corresponding data, supplementing missing position coordinate points between adjacent time stamps, and generating a position movement signal with a continuous time sequence.

During the actual acquisition, repeated recordings may occur due to various factors. Therefore, the stored corresponding data needs to be subjected to a deduplication operation, and the associated data of the repeated time stamp and the coordinate position is removed. After the deduplication is completed, there may be a situation in which a position coordinate point is missing between adjacent timestamps. In order to make the position moving signal have a continuous time sequence, linear interpolation processing can be performed on the corresponding data after the duplication removal. And estimating coordinate points of the missing positions by utilizing a linear relation through analyzing coordinate positions corresponding to adjacent time stamps. For example, given that the coordinates corresponding to the time T1 are (X1, Y1), the coordinates corresponding to the time T3 are (X3, Y3), and the coordinates of the time point are missing while the time T2 is between T1 and T3, the coordinates (X2, Y2) corresponding to the time T2 can be estimated by linear interpolation according to the coordinates of T1 and T3 and the time interval. Through the above processing, a position movement signal having a continuous time series is generated.

And step S112, acquiring contact time, contact force and contact position data of a user when the user takes the commodity through a pressure sensor of the commodity display rack, and generating a commodity contact signal.

The pressure sensor is arranged on the commodity display rack, and when a user picks up the commodity, the pressure sensor senses the change of the commodity state and starts recording related data. The sensor records the starting time and the ending time of the user contacting the commodity, and the contact time can be calculated through the two time points. Meanwhile, the pressure sensor can measure the pressure applied by a user to the commodity, and the pressure sensor is used as a measurement index of the contact force. In addition, the sensor may also determine the specific location where the user is taking the merchandise, i.e., the contact location. For example, when a user picks up a commodity on a display rack, the pressure sensor records the contact start time as T4 and the contact end time as T5, and the contact duration is obtained by calculating T5-T4. The sensor measures the pressure applied by the user as P1 as the contact force. And the position where the user takes the commodity can be positioned at the nth position of the mth layer of the display rack.

Step S113, collecting the user voice signal and the frequency, amplitude and duration data of the user voice signal through a microphone array distributed in the interaction area, and generating a voice interaction signal.

The microphone array is distributed in various interaction areas of the store, such as a promotional area, a vicinity of a counsel desk, etc. When the user communicates in these areas, the microphone array captures the user's voice signals. Meanwhile, the collected voice signals can be analyzed, and data such as frequency, amplitude and duration of the voice signals can be extracted. For example, the user inquires about discount information of goods in a promotion area, the microphone array collects voice signals of the user, and the voice signals are obtained through analysis, the frequency range of the voice signals is between F1 and F2, the amplitude is A1, and the duration is T6.

And step S114, performing spatial interpolation processing on the coordinate points of the position moving signals, performing normalization processing on the contact force data of the commodity contact signals, and performing denoising processing on the voice interaction signals to respectively obtain the position moving signals after the spatial interpolation processing, the commodity contact signals after the normalization processing and the voice interaction signals after the denoising processing.

For coordinate points of the position moving signal, since the actually acquired coordinate points may not be distributed uniformly enough, in order to make the position information spatially more continuous and accurate, a spatial interpolation process may be performed. And estimating coordinate values of the position which is not acquired by using a spatial interpolation algorithm through analyzing the distribution condition of the existing coordinate points, so as to obtain a position movement signal after the spatial interpolation processing.

For the contact force data of commodity contact signals, the acquired contact force data range is large because of different factors such as the weight, the material and the like of different commodities. The contact force data may be normalized for ease of subsequent processing and analysis. The contact force data is mapped to a specific range, for example, the contact force data is mapped to between 0 and 1, so that the contact force data of different commodities are comparable, and a commodity contact signal after normalization processing is obtained.

For voice interaction signals, the voice interaction signals may be interfered by environmental noise in the acquisition process. In order to improve the quality of the voice signal, denoising processing can be performed on the voice interaction signal. And removing the influence of environmental noise by adopting methods such as filtering, noise reduction algorithm and the like, and only retaining the voice information of the user to obtain the voice interaction signal after the denoising treatment.

And step S115, arranging the position moving signal after the spatial interpolation processing, the commodity contact signal after the normalization processing and the voice interaction signal after the denoising processing according to the time stamp sequence to form an interaction signal sequence with time sequence continuity.

After the processing of the position moving signal, the commodity contact signal and the voice interaction signal is completed, the signals can be arranged according to the sequence of the time stamps. And extracting time stamps in the position moving signal after the spatial interpolation processing, the commodity contact signal after the normalization processing and the voice interaction signal after the denoising processing, and sequencing the signals according to time. For example, the first-occurring position movement signal, the commodity contact signal, and the voice interaction signal are arranged in the front, and the second-occurring position movement signal is arranged in the rear. Through the arrangement, an interactive signal sequence with time sequence continuity is formed, and the sequence can clearly reflect the interactive behaviors of users at different times in a store.

And step S120, performing space-time feature coding processing on the interactive signal sequence through a space-time feature coder to generate a user intention feature vector, wherein the user intention feature vector comprises a requirement explicit feature, a requirement implicit feature and a requirement association feature.

After the user interaction signal sequence is obtained, the user interaction signal sequence can be input into a space-time feature encoder for space-time feature encoding processing. The space-time feature encoder is composed of a plurality of layers, each layer has different functions, and finally generates a user intention feature vector containing a requirement explicit feature, a requirement implicit feature and a requirement association feature by gradually processing the interactive signal sequence.

Step S121, inputting the interactive signal sequence into a time sequence coding layer of the space-time characteristic coder, and extracting the moving track characteristics of the position moving signal, the contact mode characteristics of the commodity contact signal and the semantic characteristics of the voice interactive signal through a long-short-term memory network.

The time sequence coding layer of the space-time characteristic coder adopts a long-short-period memory network (LSTM). When the interactive signal sequence is input to the layer, the LSTM processes the position movement signal, the commodity contact signal, and the voice interactive signal, respectively. For the position movement signal, the LSTM analyzes the coordinate change on the time sequence of the position movement signal, and extracts the movement track characteristics of the user. By correlating and analyzing the coordinate positions of different time points, the information such as the moving path, the stay area and the like of the user in the store can be known. For commodity contact signals, LSTM analyzes the contact time length, the contact force and the change rule of the contact position along with time, and the contact mode characteristics are extracted. For example, a time interval at which a user touches a commodity a plurality of times, a trend of change in touch force, or the like is analyzed. For the voice interaction signal, the LSTM can understand and analyze the semantics of the voice signal and extract the semantic features. Through the operations of word segmentation, part-of-speech tagging, semantic analysis and the like on the voice content, the intention and the requirement expressed by the user are understood.

Step S122, inputting the movement track features, the contact mode features and the semantic features output by the time sequence encoding layer into a space encoding layer of the space-time feature encoder, and extracting the distribution relation features of the movement track features, the contact mode features and the semantic features in store space through a convolutional neural network.

After the time sequence encoding layer outputs the moving track feature, the contact mode feature and the semantic feature, the moving track feature, the contact mode feature and the semantic feature can be input into a space encoding layer of the space-time feature encoder. The spatial coding layer employs a Convolutional Neural Network (CNN). The CNN performs convolution operation on the input features and analyzes the distribution relation of the movement track features, the contact mode features and the semantic features in store space. For example, the relation between the movement track of the user and the commodity display position is studied, and the relation between the contact pattern features and the commodity area and the distribution condition of the semantic features in different spatial positions are analyzed. And extracting distribution relation features of the movement track features, the contact mode features and the semantic features in store space through convolution operation.

And step 123, inputting the distribution relation features output by the space coding layer into an intention extraction layer of the space-time feature coder, and calculating the contribution weights of the movement track features, the contact mode features and the semantic features to the requirements of the user through an attention mechanism.

After the spatial encoding layer outputs the distribution relation feature, it may be input to an intention extraction layer of the spatial feature encoder. The intention extraction layer calculates the contribution weights of the movement track features, the contact pattern features and the semantic features to the user requirements by using an attention mechanism.

Step S1231, performing dimension alignment processing on the distribution relation features output by the space coding layer and the movement track features, the contact mode features and the semantic features output by the time sequence coding layer, and performing feature splicing operation on the aligned distribution relation features, the movement track features, the contact mode features and the semantic features to generate a joint feature matrix containing space association information and time sequence features.

Firstly, dimension alignment processing is required to be carried out on the distribution relation characteristics output by the space coding layer, the movement track characteristics, the contact mode characteristics and the semantic characteristics output by the time sequence coding layer. Since the dimensions of the different features may be different, in order to enable an efficient stitching operation, the dimensions of the features need to be adjusted to be consistent. And after the dimension alignment is completed, carrying out feature splicing operation on the distribution relation features, the moving track features, the contact mode features and the semantic features. And arranging and combining the distribution relation features, the movement track features, the contact mode features and the semantic features according to a set sequence to form a joint feature matrix containing space association information and time sequence features. The joint feature matrix integrates information in two dimensions, space and time.

Step S1232, calculating the association degree between each feature dimension in the joint feature matrix and the user requirement through the attention calculating unit of the intention extraction layer, namely inputting the joint feature matrix into a full-connection layer to generate a query vector, taking the moving track feature, the contact mode feature and the semantic feature as key vector and value vector respectively, and calculating the dot product similarity between the query vector and each key vector.

The attention calculating unit of the intention extraction layer processes the joint feature matrix. First, the joint feature matrix is input into the full-connection layer, and a query vector is generated through linear transformation of the full-connection layer. The query vector is used for measuring the association degree of each feature dimension in the joint feature matrix and the user requirement. Then, the movement track feature, the contact pattern feature and the semantic feature are respectively used as a key vector and a value vector. And determining the relevance of each feature dimension to the user requirement by calculating the dot product similarity of the query vector and each key vector. The higher the dot product similarity, the tighter the association of the feature dimension with the user's needs.

And S1233, carrying out softmax normalization processing on the dot product similarity value to generate local attention weights reflecting the attention degree of each characteristic dimension to the user requirement.

After obtaining the dot product similarity values of the query vector and each key vector, in order to make the similarity values have comparability and can represent the attention degree of each feature dimension to the user requirement, the dot product similarity values can be subjected to softmax normalization processing. The Softmax normalization process converts the dot product similarity values to a probability distribution such that the sum of the weights for all feature dimensions is 1. After normalization processing, the obtained local attention weight can accurately reflect the attention degree of each feature dimension to the user demand, and the higher the weight is, the more important the feature dimension is in user demand analysis.

Step S1234, the local attention weights are weighted and adjusted by combining the spatial association strength of each feature in the distribution relation features, and global contribution weights fusing the spatial importance are generated.

After the local attention weight is obtained, the spatial association strength of each feature in the distribution relation features also needs to be considered. The distribution relation features reflect the distribution of the features in store space, and the influence of the features of different spatial positions on the user needs can be different. Therefore, the local attention weight can be weighted and adjusted by combining the spatial association strength of each feature in the distribution relation features. For the features with higher spatial correlation strength, the corresponding local attention weight can be increased, and for the features with lower spatial correlation strength, the weight can be properly reduced. Through the weighting adjustment, global contribution weights fusing the spatial importance are generated, so that the final weights can more accurately reflect the comprehensive contribution of each feature to the user demand.

Step S1235, the global contribution weights are respectively related to the movement track characteristics, the contact mode characteristics and the semantic characteristics.

After the global contribution weights are generated, the weights may be associated to the movement trajectory features, the contact pattern features, and the semantic features, respectively. Each feature corresponds to a global contribution weight that represents the importance of the feature in the user demand analysis. By associating global contribution weights, features that have a greater impact on user demand can be highlighted.

And step S124, carrying out weighted fusion on the track characteristics, the contact mode characteristics and the semantic characteristics according to the contribution weight, and generating a demand explicit characteristic reflecting the clear expression demand of the user.

After global contribution weights corresponding to the movement track features, the contact mode features and the semantic features are obtained, the features can be weighted and fused according to the contribution weights. For each feature, it is multiplied with a corresponding global contribution weight, and then the weighted results of all features are combined. Through the weighted fusion operation, the explicit demand characteristic reflecting the explicit demand of the user is generated. For example, if the movement track features show that the user stays in a certain commodity area for many times, the contact mode features indicate that the user has frequent contact with the commodity, the user explicitly inquires about related information of the commodity in the semantic features, and the global contribution weights of the features are higher, then after weighted fusion, the generated demand explicit features can clearly reflect the demand of the user on the commodity.

And step S125, extracting the demand trend which is not explicitly expressed by the user through the hidden layer neuron activation value of the intention extraction layer, and generating the demand implicit characteristic.

The hidden layer of the intention extraction layer can output a set of neuron activation values when processing the multi-modal interaction signal sequence, wherein the set of activation values comprises nonlinear transformation results of each neuron on the input characteristics. By analyzing and processing the activation value sets, the demand trend which is not explicitly expressed by the user can be extracted, and further, the demand implicit characteristic is generated.

Step S1251, acquiring a neuron activation value set output by an implicit layer of the intent extraction layer when the multi-mode interaction signal sequence is processed, wherein the activation value set is composed of nonlinear transformation results of each neuron on input characteristics.

When a multimodal interaction signal sequence is input to an underlying layer of the intent extraction layer, neurons of the underlying layer perform a nonlinear transformation on the input features. Each neuron calculates the input characteristics according to its own weight and bias to obtain an output value, i.e. a neuron activation value. The activation values of all neurons constitute a set of neuron activation values. The set of neuron activation values contains complex processing results of hidden layers on input features and deep information of user behavior patterns.

And step 1252, performing characteristic dimension reduction processing on the activation value set, screening out an activation value subset with differentiation degree to the user requirement through a characteristic selection algorithm, and retaining main information reflecting the user deep behavior mode in the activation value set.

Since the set of neuron activation values may contain a large number of feature dimensions, some of which are less differentiated from the user's needs and thus interfere with subsequent analysis. Thus, feature dimension reduction processing can be performed on the activation value set. And screening out the subset of the activation values with differentiation to the user demands by a feature selection algorithm, such as a method based on correlation analysis, information gain and the like. During the screening process, the main information reflecting the deep behavior pattern of the user can be kept in focus. For example, some neuron activation values are closely related to a particular behavior pattern of the user, while others may be random noise or less discriminating between user needs, the former being retained by the feature selection algorithm, the latter being removed.

Step S1253, inputting the activation value subset into a cluster analysis module, and identifying distribution cluster groups of the activation values through a density clustering algorithm, wherein each distribution cluster group corresponds to a demand pattern which is not explicitly expressed.

After the subset of activation values is obtained, it may be input to a cluster analysis module. The cluster analysis module adopts a density clustering algorithm, such as a DBSCAN algorithm. The algorithm divides the activation values into different clusters according to the density distribution of the activation values. Each cluster represents a set of activation values with similar characteristics, corresponding to a demand pattern that is not explicitly expressed. For example, certain clusters of activation values may represent user interest in certain potential merchandise, which, although not explicitly expressed by the user during interaction, can be found by cluster analysis of activation values.

And step 1254, extracting the feature center vector of each distributed cluster, and establishing a mapping relation between cluster features and non-explicit requirements by combining the non-satisfied requirement records in the historical user interaction data set.

After the distributed clusters of activation values are identified, feature center vectors for each cluster may be extracted. The feature center vector represents the dominant feature of the cluster. Then, in combination with the unmet need record in the historical user interaction dataset, the association of the characteristics of each cluster with the historically unmet need is analyzed. Through the analysis, a mapping relation between cluster characteristics and non-explicit requirements is established. For example, if the feature center vector of a cluster has similarity with the historically unmet needs of the user for a certain class of merchandise, then a mapping relationship between the cluster features and the inexact needs of the class may be established.

And step 1255, screening the frequent item mapping relation appearing in the historical prior data to be used as the demand trend which is not explicitly expressed by the user.

After the mapping relations between cluster characteristics and non-explicit requirements are established, the mapping relations can be screened. In the historical prior data, a large amount of user interaction information and unsatisfied demand conditions can be recorded. And (3) through statistics and analysis of the historical data, mapping relations with higher occurrence frequency, namely frequent item mapping relations, are found out. These frequent item mappings represent non-explicit demand patterns that often occur in past interactions by users, with high reliability and representativeness. The frequent item mapping relations are used as demand trends which are not explicitly expressed by users, so that potential demands of the users can be captured more accurately.

Step S1255, mapping the demand trend to a natural language description space through a semantic conversion model, and generating a demand implicit characteristic with semantic interpretability.

After obtaining the demand trends that are not explicitly expressed by the user, these demand trends can be mapped to natural language description space through a semantic conversion model in order to make them easier to understand and apply. The semantic conversion model performs semantic analysis and conversion on demand trends and converts the demand trends into description forms of natural language. For example, if demand tends to indicate that a user is potentially interested in a particular functional commodity, the semantic conversion model will describe it as "the user may be interested in a commodity that is XX functional". The thus generated demand implicit features have semantic interpretability that enables store personnel or systems to better understand the potential demands of the user.

And S126, analyzing the association relation between the demand explicit feature and the demand implicit feature, extracting the extending direction of the user demand, and generating the demand association feature.

In order to more fully understand the needs of users, it is necessary to analyze the association between explicit and implicit needs features. Through the analysis, the extending direction of the user requirement can be extracted, and further the requirement correlation characteristic is generated.

Step S1261, constructing a demand feature correlation matrix, wherein row vectors of the demand feature correlation matrix are semantic representations of demand explicit features, and column vectors are semantic representations of the demand implicit features.

First, a demand feature correlation matrix may be constructed. The row vectors of the demand feature correlation matrix are composed of semantic representations of demand explicit features, and the column vectors are composed of semantic representations of demand implicit features. The semantic representation of demand explicit features and demand implicit features is the transformation of these features into a vector form with semantic meaning. For example, the explicit demand feature "user explicitly indicates that a cell phone is required to be purchased" may be converted into a vector containing semantic information related to the cell phone, and the implicit demand feature "user may be interested in a cell phone accessory" may be converted into a vector containing semantic information related to the cell phone accessory. The vectors are arranged in rows and columns to form the demand feature correlation matrix.

Step S1262, calculating the semantic relevance of each pair of features in the demand feature relevance matrix, and screening feature pairs with the semantic relevance exceeding a preset threshold as significant relevance feature pairs.

After the demand feature correlation matrix is built, the semantic relevance of each pair of features in the demand feature correlation matrix can be calculated. The semantic relevance represents a semantic similarity or relevance between the demand explicit feature and the demand implicit feature. Semantic relevance may be calculated using methods based on semantic networks, word vector similarity, and the like. For example, semantic relevance is measured by computing cosine similarity between the demand explicit feature vector and the demand implicit feature vector. Then, a preset threshold value can be set, feature pairs with semantic relevance exceeding the threshold value are screened out, and the feature pairs are used as significant relevance feature pairs. These significant association feature pairs indicate that there is a strong association between the demand explicit feature and the demand implicit feature.

And step S1263, analyzing the appearance sequence of the obvious association characteristic pairs in the historical interaction data set, counting the appearance frequency of the implicit demand characteristic in the subsequent interaction after the appearance of the explicit demand characteristic, and determining the time sequence dependency frequency of the demand extension.

After the salient associated feature pairs are obtained, the order of occurrence of the salient associated feature pairs in the historical interaction dataset may be analyzed. The historical interaction data set records interaction processes and demand change conditions of a large number of users. Through analysis of the data, the occurrence frequency of the implicit demand features in subsequent interactions after the occurrence of the explicit demand features is counted. For example, if in the historical data, after the user explicitly indicates that a handset is required to be purchased, interest in the handset accessories is often expressed in subsequent interactions, the frequency of occurrence of such a requirement extension may be counted. The frequency of occurrence is taken as the time-dependent frequency of demand extension, which reflects the likelihood and regularity of demand extension from explicit to implicit.

Step S1264, constructing a demand extension model based on the time sequence dependent frequency, wherein the demand extension model takes an explicit demand characteristic as a starting point, takes an implicit demand characteristic as an ending point, and takes the time sequence dependent frequency as a weight of a side to form a demand extension relation diagram.

A demand extension model can be constructed based on the time-dependent frequency of demand extension. The demand extension model starts with explicit demand features and ends with implicit demand features, with timing dependent frequencies as the weights of the edges. By the method, a requirement extension relation graph is formed. In the demand extension relation diagram, nodes represent demand characteristics, edges represent extension relations among demands, and weights of the edges represent the possibility of extending the demands. For example, if the timing dependency frequency between the explicit demand feature "buy cell phone" and the implicit demand feature "buy cell phone accessory" is high, then the edge connecting the two nodes in the demand extension graph is weighted more, indicating a greater likelihood of extending from buying cell phone demand to buying cell phone accessory demand.

Step S1265, identifying a target extension path starting from the explicit demand characteristics through the path analysis function of the demand extension relation diagram, wherein the implicit demand characteristics corresponding to the destination extension path end point are extension directions of user demands, and the weight of the edges of the target extension path is larger than the set weight.

The demand extension relation diagram has a path analysis function. By the path analysis function, a target extension path meeting the set condition can be searched from the explicit demand characteristics. Setting a weight threshold value, and screening out paths with the weights of edges larger than the weight threshold value as target extension paths. The implicit demand feature corresponding to the end point of the target extension path is the extension direction of the user demand. For example, starting from the explicit requirement feature "buy cell phone", a path with an edge weight greater than the set weight is found in the requirement extension relation diagram, and the end point of the path is the implicit requirement feature "buy cell phone protective shell", and then "buy cell phone protective shell" is the direction in which the user requirement extends from the requirement of buying cell phone.

Step S1266, carrying out structural representation on the extending direction, and generating a requirement-related feature comprising an extending starting point, an extending path and an extending end point, wherein the structural representation is used for describing a specific path of extending the user requirement from the explicit expression to the potential requirement.

After determining the direction of extension of the user's needs, the direction of extension may be represented structurally. The demand-related features may include information such as an extension start point, an extension path, and an extension end point. The extension starting point is an explicit demand feature, the extension path is a specific path from the starting point to the end point in the demand extension relation graph, and the extension end point is an implicit demand feature. By the above structured representation, the specific path that the user needs extend from explicit representation to potential needs can be clearly described. For example, the requirement association feature may be expressed as "from purchasing a handset requirement, through an intermediate requirement related to the handset accessory, to purchasing a handset protective case requirement".

And S127, splicing the demand explicit feature, the demand implicit feature and the demand association feature to generate a user intention feature vector.

After the demand explicit feature, the demand implicit feature and the demand associated feature are obtained, the demand explicit feature, the demand implicit feature and the demand associated feature can be spliced. The splicing mode is to arrange and combine the demand explicit feature, the demand implicit feature and the demand association feature according to a set sequence to form a user intention feature vector. The user intention feature vector integrates information such as requirements, potential requirements, extending relations of requirements and the like which are explicitly expressed by the user, and can reflect the intention of the user more comprehensively. For example, the demand explicit feature vector, the demand implicit feature vector, and the demand associated feature vector are sequentially spliced together to form a longer vector as the user intention feature vector.

And step S130, acquiring the current interactive environment characteristics of the store by using a scene characteristic extractor, wherein the interactive environment characteristics comprise commodity space distribution characteristics, equipment state characteristics and time sequence flow characteristics.

In order to make personalized recommendations better, it is necessary to obtain the current interactive environmental characteristics of the store. The scene feature extractor collects and processes data from different aspects to generate interaction environment features including merchandise spatial distribution features, device status features, and timing traffic features.

Step S131, acquiring coordinate distribution data of a commodity display rack through a store electronic map system, extracting distance data of the space position of each commodity and adjacent commodities, and generating commodity space distribution characteristics.

The store electronic map system records detailed coordinate distribution information of the commodity display rack. By the store electronic map system, specific coordinate positions of each commodity display rack in the store can be obtained. Then, based on these coordinate positions, the spatial positions of the respective commodities and their distance relationships with the adjacent commodities are analyzed. For each commodity, its specific location coordinates within the store may be determined and its distance from the surrounding adjacent commodity calculated. For example, the coordinates of commodity a are (Xa, ya), the coordinates of adjacent commodity B are (Xb, yb), and the distance between commodity a and commodity B can be obtained by calculating a distance formula between two points. And (5) sorting and storing the spatial positions and the adjacent distance data of the commodities to generate commodity spatial distribution characteristics.

And S132, acquiring the online state, screen brightness and speaker volume data of the interactive terminal from a store equipment management system, and generating equipment state characteristics.

The store equipment management system is responsible for managing and monitoring various interactive terminal equipment in the store. By the system, the online state of the interactive terminal, the screen brightness, the loudspeaker volume and other data can be obtained. The online state of the interactive terminal is divided into online and offline states, and the state information of each terminal is recorded in real time. The screen brightness and speaker volume data reflect the current operating parameters of the terminal device. For example, the on-line state of the interactive terminal C is on-line, the screen brightness is a certain set value, and the speaker volume is another set value. And integrating the online state, the screen brightness and the loudspeaker volume data to generate the equipment state characteristics.

Step S133, counting the triggering times of the position moving signals in the current time window, calculating the number of users entering the store in unit time and the user density data of each area, and generating time sequence flow characteristics.

In order to obtain the time sequence flow characteristics, statistics and analysis are required to be carried out on the position movement signals in the current time window.

Step S1331, determining the starting time and the ending time of the current time window, and extracting the data of the time stamp in the time window in the position moving signal.

First, a current time window may be determined, with its start time and end time set. For example, the start time is Tstart and the end time is Tend. The data with the time stamp within the time window is then screened from the previously acquired position movement signal. These time-conditioned data are extracted for subsequent statistical analysis.

And S1332, counting the number of position coordinate points appearing for the first time in the data, and taking the position coordinate points as the number of users entering a store in unit time.

After the position moving signal data in the time window is extracted, the number of position coordinate points which occur for the first time can be counted. When a user first enters a store, a new location coordinate point may be triggered. By counting the number of these first-occurring position coordinate points, the number of users who enter the store per unit time can be approximated. For example, in the time window Tstart-Tend, it is found that new position coordinate points P1, P2, P3, etc. first appear, and then the number of these coordinate points represents the number of users who enter the store in a unit time.

And S1333, dividing the store into a plurality of regional units with equal areas, and counting the number of position coordinate points in each regional unit.

To calculate the user density for each zone, the store may be divided into a plurality of equal area zone units. The division mode of the area units can be reasonably determined according to the actual layout and the size of the store. For example, stores are divided according to set grids, and each grid is an area unit. Then, the number of position coordinate points in each area unit is counted. And screening and counting the coordinate data in the position moving signals to obtain the number of position coordinate points in each area unit.

Step S1334, calculating the ratio of the number of the position coordinate points of each area unit to the area of the area unit as the user density data of the area.

After the number of position coordinate points in each area unit is obtained, the ratio of the number to the area of the area unit can be calculated. The ratio is the user density data for that region. The user density data reflects the user's density in each region. For example, the area of the area unit a is S, where the number of position coordinate points is N, and then the user density data of the area is N/S.

And S1335, carrying out logarithmic transformation processing on the user quantity and the user density data of each area to generate time sequence flow characteristics.

In order to make the data more in line with the input requirement of the model, the user quantity entering the store in unit time and the user density data of each area can be subjected to logarithmic transformation. The logarithmic transformation can adjust the distribution of the data to make it more stable and comparable. After logarithmic transformation, the processed data are integrated together to generate time sequence flow characteristics.

And step S134, carrying out standardization processing on the distance data of the commodity space distribution characteristics, carrying out normalization processing on the brightness and volume data of the equipment state characteristics, and carrying out sliding window smoothing processing on the user density data of the time sequence flow characteristics to obtain the processed commodity space distribution characteristics, the equipment state characteristics and the time sequence flow characteristics.

Because the data ranges and distributions of different features may be widely different, in order to ensure the accuracy and effectiveness of subsequent processing, the feature data needs to be processed correspondingly.

The distance data of the commodity space distribution feature can be subjected to standardization processing. The normalization process is to convert the distance data into a distribution with a mean value of 0 and a standard deviation of 1. Through the processing, the dimension difference of the distance data between different commodities can be eliminated, so that the data has comparability. For example, the distance data between the commodity a and the commodity B, the distance data between the commodity C and the commodity D, and the like are normalized to obtain a set of normalized distance data.

Normalization processing may be performed on the luminance and volume data of the device state features. Normalization is the mapping of data to a specific range, typically between 0 and 1. Through normalization processing, brightness and volume data of different devices can be unified into a standard range, so that subsequent analysis and comparison are facilitated. For example, the screen brightness and speaker volume data of different interactive terminals are normalized to obtain a group of normalized brightness and volume data.

For user density data of the time series flow characteristic, sliding window smoothing processing can be performed. Sliding window smoothing is the process of sliding a window of a fixed size over a sequence of data, and averaging or other statistical processing of the data within the window to reduce fluctuations and noise in the data. For example, a sliding window with a size of M is used to smooth the user density data sequence, so as to obtain a set of smoothed user density data.

And S135, carrying out feature fusion on the processed commodity spatial distribution features, the equipment state features and the time sequence flow features according to the preset dimension positions to generate interaction environment features.

After the processing of the spatial distribution characteristics, the equipment state characteristics and the time sequence flow characteristics of the commodity is completed, the spatial distribution characteristics, the equipment state characteristics and the time sequence flow characteristics can be subjected to characteristic fusion according to the preset dimension positions. The preset dimension position is predetermined according to the design and the requirement of the model. And arranging and combining the three processed features according to the preset sequence and mode to form a new feature, namely the interactive environment feature. For example, the processed commodity space distribution feature vector, the equipment state feature vector and the time sequence flow feature vector are spliced together in sequence to form a longer vector which is used as the interaction environment feature.

And step 140, carrying out feature fusion reasoning processing on the user intention feature vector and the interaction environment feature through a recommendation decision network to generate a personalized recommendation instruction, wherein the personalized recommendation instruction comprises a content identifier, a trigger position and a response mode.

The recommendation decision network comprehensively processes the user intention feature vector and the interaction environment feature to generate a personalized recommendation instruction.

And step S141, inputting the user intention feature vector and the interaction environment feature into a feature fusion layer of the recommendation decision network, and screening feature dimensions contributing to recommendation decisions through a gating mechanism to generate a fusion feature vector.

The feature fusion layer of the recommendation decision network receives user intention feature vectors and interaction environment features. In this layer, gating mechanisms can be employed to screen feature dimensions that contribute to recommendation decisions. The gating mechanism evaluates and screens the input feature vectors. For example, for each dimension in the user intent feature vector and the interaction environment feature vector, the gating mechanism may calculate its importance score for the recommendation decision, the higher scoring dimension may be preserved, and the lower scoring dimension may be discarded. And combining the reserved characteristic dimensions together through the screening operation to generate a fusion characteristic vector.

And S142, carrying out spatial position prediction on the fusion feature vectors by utilizing a position reasoning sub-network of the recommendation decision network, calculating the probability value of the user staying in each region, and selecting the region with the highest probability value as a trigger position.

The position reasoning sub-network of the recommendation decision network can process the fusion feature vector to predict probability values of stay of the user in each region of the store. The position reasoning sub-network learns the relation between the behavior mode of the user and the environment characteristics, and calculates the possibility of the user staying in each area according to the information in the fused characteristic vector. For example, by analyzing information such as a movement track of a user, a commodity contact condition, and environmental characteristics of a store, it is predicted that the probability of a user staying in an area a of the store is Pa, and the probability of a user staying in an area B is Pb. Then, the area with the highest probability value is selected as the trigger position. If Pa > Pb, then the A region will be selected as the trigger position.

And S143, extracting a keyword vector of the explicit feature required in the user intention feature vector and an attribute feature vector of the commodity sold in a store through a content matching subnetwork of the recommendation decision network, calculating a cosine similarity value of the keyword vector of the user intention feature vector and the attribute feature vector of each commodity, and taking the corresponding commodity with the cosine similarity value larger than a set threshold value as a content identifier.

The content matching sub-network of the recommendation decision network may extract keyword vectors requiring explicit features from the user intent feature vectors. The demand explicit feature contains demand information explicitly expressed by a user, and keyword vectors are obtained by processing and extracting the information. At the same time, attribute feature vectors of the merchandise being sold at the store may be obtained, which describe various attributes of the merchandise, such as function, brand, price, etc. And then, calculating cosine similarity values of the keyword vectors of the user intention feature vectors and the attribute feature vectors of the commodities. The cosine similarity value indicates the degree of similarity between the two vectors, with larger values indicating more similarity. Setting a threshold value, and taking the corresponding commodity with the cosine similarity value larger than the threshold value as the content identifier. For example, if the cosine similarity value of the keyword vector of the user intention feature vector and the attribute feature vector of the commodity E is greater than the set threshold, the commodity E will be identified as content.

Step S144, based on the response adaptation sub-network of the recommendation decision network and the screen brightness and speaker volume data of the equipment state characteristics, a presentation mode adapted to the current equipment state is selected as a response mode.

The response adaptation sub-network of the recommendation decision network combines the screen brightness and the loudspeaker volume data of the equipment state characteristics to select a proper presentation mode as a response mode. Different equipment states are suitable for different presentation modes, for example, when the screen brightness is higher, a display mode with bright color and high contrast ratio is suitable, and when the loudspeaker volume is higher, a voice playing mode with clear sound and moderate volume is suitable. The response adaptation sub-network can select the optimal adaptation mode from a preset presentation mode list as a response mode according to the screen brightness and the loudspeaker volume data of the equipment. For example, if the screen brightness of the current device is high, the response adaptation sub-network may select a vivid picture display mode as the response mode.

And S145, carrying out consistency check on the probability value of the triggering position, the matching degree of the content identifier and the matching degree of the response mode, and carrying out instruction encapsulation on the content identifier, the triggering position and the response mode which pass the check to generate a personalized recommendation instruction.

After the trigger location, content identification and response mode are determined, they may be checked for consistency. The consistency check is to check whether the probability value of the trigger position, the matching degree of the content identification and the matching degree of the response mode are mutually coordinated and reasonable. For example, if the probability value of the trigger location is high, it is checked whether the corresponding content identifier is related to the goods or services at the location, and whether the response mode is suitable for presentation to the user at the location. If there is a significant discrepancy or irrational situation between these values, the verification is not passed.

In the verification process, the probability value of the trigger position, the matching degree of the content identification and the matching degree of the response mode can be comprehensively evaluated. For the probability value of the trigger position, it can be considered whether it is within a reasonable range and matches the user behavior pattern in the history data. The matching degree of the content identification can be judged according to the similarity between the user intention feature vector and the commodity attribute feature vector, so that the recommended commodity is ensured to be really related to the requirement of the user. The adaptation degree of the response mode is combined with the device state characteristics, such as screen brightness and speaker volume, so as to judge whether the response mode can effectively convey information to the user in the current device state.

If the verification is passed, the content identification, the trigger position and the response mode can be packaged in an instruction mode. Instruction encapsulation is the combining of such information into a complete instruction structure so that subsequent systems can execute accurately. The instruction structure may include detailed information of content identification, such as the name, number, etc. of the commodity, specific coordinates or area identification of the trigger position, and specific description of the response mode, such as displaying a picture, playing voice, etc. The packaged instruction is a personalized recommendation instruction, and personalized commodity recommendation service can be provided for the user according to the intention of the user and the environmental characteristics of a store.

And step S150, executing the personalized recommendation instruction and collecting a user response signal, wherein the user response signal is used for updating the coding parameters of the space-time characteristic encoder and the inference weight of the recommendation decision network.

After the personalized recommendation is generated, the personalized recommendation can be executed and the response signals of the user can be collected so as to optimize and improve the system.

And step S151, presenting commodity information of the content identifier at the triggering position of the personalized recommendation instruction through corresponding equipment, and triggering user interaction according to the response mode.

According to the personalized recommendation instruction, commodity information of the content identifier can be presented through corresponding equipment at the designated trigger position. If the trigger location is an area of a store, the area may be equipped with an electronic display, smart shelf, etc. The system will control these devices to display merchandise information in a responsive manner. For example, if the response mode is to display a picture, the device will display the picture of the commodity on the screen and possibly carry information such as the introduction and price of the commodity, and if the response mode is to play voice, the device will play the relevant voice introduction of the commodity through the loudspeaker. By triggering the interaction between the user and the recommended commodity in the mode, the attention of the user is attracted, and the user is prompted to further know the commodity.

Step S152, collecting stay time data of a user at a trigger position through a sensor array of a position moving signal, collecting contact frequency data of the user on a recommended commodity through a pressure sensor of a commodity contact signal, collecting feedback voice data of the user on recommended content through a microphone array of a voice interaction signal, and generating a user response signal.

To learn the user's response to the recommended content, the user's response data may be collected by a variety of sensors. The sensor array of the position movement signal can continuously monitor the stay condition of the user at the trigger position, and record the time from entering the position to leaving the position, so as to obtain stay time data. The stay length data may reflect the user's interest in the recommended content, and the longer the stay time, the more likely the user is interested in the recommended merchandise.

The pressure sensor of the commodity contact signal is installed on a display rack of the recommended commodity, and when a user contacts the recommended commodity, the sensor records the number of times of contact. The contact number data can represent the actual attention and operation condition of the user on the commodity, and the larger contact number can mean that the user has potential willingness to purchase.

The microphone array of the voice interaction signal can collect feedback voice data of the recommended content at the trigger position. The user may issue a rating, a question, etc. on the recommended goods, and these voice information can directly reflect the user's ideas and attitudes.

And integrating the stay time data, the contact frequency data and the feedback voice data together to generate a user response signal.

And step 153, inputting the user response signals into a parameter updating module of the space-time characteristic encoder, calculating output errors of a time sequence coding layer, a space coding layer and an intention extraction layer, and adjusting weight parameters of each layer through a back propagation algorithm.

The user response signal is input to a parameter update module of the space-time feature encoder. The module calculates the output errors of the temporal coding layer, the spatial coding layer and the intention extraction layer. The output error refers to the difference between the output result of the model and the actual response of the user. For example, the motion trajectory characteristics output by the timing encoding layer may generate errors if they do not match the actual dwell behavior of the user at the trigger position.

To reduce these errors, a back propagation algorithm may be used. The back propagation algorithm is an optimization algorithm based on gradient descent, which starts from the output layer of the model, propagates the error back to each layer, and calculates the contribution degree of the weight parameter of each layer to the error. And according to the calculation result, adjusting the weight parameters of each layer, so that the output result of the model is more similar to the actual response of the user. By continuously and iteratively updating the weight parameters, the space-time feature encoder can better capture the interactive signals of the user and improve the accuracy of encoding.

And step 154, inputting the user response signals into a weight optimization module of the recommendation decision network, analyzing the deviation value of the matching degree of the recommended content and the feedback of the user, and optimizing the inference weights of the position inference sub-network, the content matching sub-network and the response adaptation sub-network through a gradient descent algorithm.

Likewise, the user response signal is also input to the weight optimization module of the recommendation decision network. The module analyzes the deviation value between the matching degree of the recommended content and the feedback of the user. The matching degree of the recommended content refers to the degree of coincidence between the recommended commodity and the user demand, and the user feedback directly reflects the actual opinion of the user on the recommended content. If the recommended items differ significantly from the needs of the user, and the user feedback also shows dissatisfaction with the recommended content, a large deviation value will result.

To reduce this bias value, a gradient descent algorithm may be used. The gradient descent algorithm calculates the gradient of the inference weight to the deviation value, and adjusts the inference weights of the position inference sub-network, the content matching sub-network and the response adaptation sub-network according to the direction and the size of the gradient. Through continuous iterative optimization, the recommendation decision network can generate personalized recommendation instructions more accurately according to user intention feature vectors and interaction environment features, and the accuracy and effectiveness of recommendation are improved.

And step S155, synchronously storing the updated space-time characteristic encoder parameters and the optimized recommended decision network weights to form an iterative optimized recommended interaction system.

After the parameter updating of the space-time feature encoder and the weight optimization of the recommendation decision network are completed, the updated space-time feature encoder parameter and the optimized recommendation decision network weight can be synchronously stored. The purpose of storing these parameters and weights is to enable the use of the latest model configuration in the subsequent recommendation process.

After each time the personalized recommendation instruction is executed and the user response signal is collected, the parameter updating and weight optimizing processes are repeated. Through continuous iterative optimization, the recommendation interaction system can gradually learn the behavior mode and preference of the user, improve the quality and accuracy of recommendation, and provide more personalized and accurate commodity recommendation service for the user.

FIG. 2 illustrates a schematic diagram of exemplary hardware and software components of a store personalized recommendation interaction system 100 incorporating user portraits that may implement the concepts of the present application, provided by some embodiments of the present application. For example, the processor 120 may be used on the store personalized recommendation interaction system 100 in conjunction with user portraits and to perform the functions of the present application.

The user portrayed store personalized recommendation interaction system 100 may be a general purpose server or a special purpose server, both of which may be used to implement the user portrayed store personalized recommendation interaction method of the present application. Although only one server is shown, the functionality described herein may be implemented in a distributed fashion across multiple similar platforms for convenience to balance processing loads.

For example, a store personalized recommendation interaction system 100 incorporating a user representation may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and a storage medium 140 of different forms, such as a disk, ROM, or RAM, or any combination thereof. Illustratively, the store personalized recommendation interaction system 100 in conjunction with a user representation may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The method of the present application may be implemented in accordance with these program instructions. The store personalized recommendation interaction system 100 in conjunction with the user representation also includes an I/O interface 150 between the computer and other input and output devices.

For ease of illustration, only one processor is depicted in the store personalized recommendation interactive system 100 in conjunction with a user representation. It should be noted, however, that the store personalized recommendation interactive system 100 in conjunction with user portraits of the present application may also include multiple processors, and thus the steps performed by one processor described in the present application may also be performed jointly by multiple processors or separately. For example, if the steps a and B are performed by a processor of the store personalized recommendation interactive system 100 in conjunction with a user representation, it should be understood that the steps a and B may be performed by two different processors together or in one processor alone. For example, the first processor performs step a, the second processor performs step B, or the first processor and the second processor together perform steps a and B.

In addition, the embodiment of the invention also provides a readable storage medium, wherein computer executable instructions are preset in the readable storage medium, and when a processor executes the computer executable instructions, the store personalized recommendation interaction method combined with the user portrait is realized.

It should be noted that in order to simplify the presentation of the disclosure and thereby aid in understanding one or more embodiments of the invention, various features are sometimes grouped together in a single embodiment, figure, or description thereof.

Claims (10)

1. A store personalized recommendation interaction method combined with a user portrait, the method comprising:

acquiring a user interaction signal sequence, wherein the interaction signal sequence comprises a position movement signal, a commodity contact signal and a voice interaction signal of a user in a store;

Performing space-time feature coding processing on the interactive signal sequence through a space-time feature coder to generate a user intention feature vector, wherein the user intention feature vector comprises a requirement explicit feature, a requirement implicit feature and a requirement association feature;

Acquiring current interaction environment characteristics of a store by using a scene characteristic extractor, wherein the interaction environment characteristics comprise commodity space distribution characteristics, equipment state characteristics and time sequence flow characteristics;

performing feature fusion reasoning processing on the user intention feature vector and the interaction environment feature through a recommendation decision network to generate a personalized recommendation instruction, wherein the personalized recommendation instruction comprises a content identifier, a trigger position and a response mode;

executing the personalized recommendation instruction and collecting a user response signal, wherein the user response signal is used for updating the coding parameters of the space-time characteristic encoder and the inference weight of the recommendation decision network.

2. The method for personalized recommended interaction of store in combination with user portraits according to claim 1, wherein the performing space-time feature encoding processing on the interaction signal sequence by a space-time feature encoder generates a user intention feature vector, the user intention feature vector comprising a demand explicit feature, a demand implicit feature and a demand associated feature, and the method comprises:

inputting the interactive signal sequence into a time sequence coding layer of the space-time characteristic coder, and extracting the moving track characteristics of the position moving signals, the contact mode characteristics of commodity contact signals and the semantic characteristics of voice interactive signals through a long-term and short-term memory network;

Inputting the moving track features, the contact mode features and the semantic features output by the time sequence coding layer into a space coding layer of the space-time feature coder, and extracting the distribution relation features of the moving track features, the contact mode features and the semantic features in store space through a convolutional neural network;

Inputting the distribution relation features output by the space coding layer into an intention extraction layer of the space-time feature coder, and calculating contribution weights of the movement track features, the contact mode features and the semantic features to user requirements through an attention mechanism;

Carrying out weighted fusion on the track features, the contact mode features and the semantic features according to the contribution weights to generate a demand explicit feature reflecting the explicit expression demand of the user;

extracting demand trends which are not explicitly expressed by a user through hidden layer neuron activation values of the intention extraction layer, and generating demand implicit characteristics;

Analyzing the association relation between the demand explicit feature and the demand implicit feature, extracting the extending direction of the user demand, and generating the demand association feature;

and splicing the demand explicit feature, the demand implicit feature and the demand association feature to generate a user intention feature vector.

3. The method for personalized recommended interaction of store in combination with user portrait according to claim 2, wherein the step of inputting the distribution relation features output by the spatial coding layer into the intention extraction layer of the space-time feature encoder, calculating the contribution weights of the movement track features, the contact pattern features and the semantic features to the user demands through an attention mechanism comprises the steps of:

Performing dimension alignment processing on the distribution relation features output by the space coding layer and the movement track features, the contact mode features and the semantic features output by the time sequence coding layer, and performing feature splicing operation on the aligned distribution relation features and the movement track features, the contact mode features and the semantic features to generate a joint feature matrix containing space association information and time sequence features;

The attention calculating unit of the intention extracting layer is used for calculating the association degree of each feature dimension in the joint feature matrix and the user requirement, specifically, the joint feature matrix is input into a full-connection layer to generate a query vector, the motion track feature, the contact mode feature and the semantic feature are respectively used as key vectors and value vectors, and the dot product similarity of the query vector and each key vector is calculated;

Carrying out softmax normalization processing on the dot product similarity value to generate local attention weights reflecting attention degrees of all feature dimensions to user demands;

Combining the spatial association strength of each feature in the distribution relation features, and carrying out weighted adjustment on the local attention weight to generate a global contribution weight fusing the spatial importance;

and respectively associating the global contribution weights to the movement track characteristics, the contact mode characteristics and the semantic characteristics.

4. The method for user portrayal-combined store personalized recommendation interaction according to claim 2, wherein the extracting a demand trend not explicitly expressed by a user through an implicit layer neuron activation value of the intention extraction layer, generating a demand implicit feature, comprises:

Acquiring a neuron activation value set output by an implicit layer of the intent extraction layer when processing a multi-mode interaction signal sequence, wherein the activation value set is composed of nonlinear transformation results of each neuron on input characteristics;

Performing feature dimension reduction processing on the activation value set, screening out activation value subsets with differentiation on user demands through a feature selection algorithm, and retaining main information reflecting a user deep behavior mode in the activation value set;

Inputting the activation value subset into a cluster analysis module, and identifying distribution cluster groups of the activation values through a density clustering algorithm, wherein each distribution cluster group corresponds to a demand mode which is not explicitly expressed;

Extracting a feature center vector of each distributed cluster, and establishing a mapping relation between cluster features and non-explicit requirements by combining the records of the unsatisfied requirements in the historical user interaction data set;

Screening a frequent item mapping relation appearing in the historical prior data to be used as a demand trend which is not explicitly expressed by a user;

mapping the demand trend to a natural language description space through a semantic conversion model, and generating a demand implicit characteristic with semantic interpretability.

5. The method for personalized recommended interaction of store in combination with user portrait according to claim 2, wherein the analyzing the association relation between the explicit requirement feature and the implicit requirement feature, extracting the extending direction of the user requirement, and generating the requirement association feature comprises:

Constructing a demand feature association matrix, wherein row vectors of the demand feature association matrix are semantic representations of demand explicit features, and column vectors are semantic representations of demand implicit features;

Calculating the semantic association degree of each pair of features in the demand feature association matrix, and screening feature pairs with the semantic association degree exceeding a preset threshold value as obvious association feature pairs;

Analyzing the appearance sequence of the obvious association feature pairs in the historical interaction data set, counting the appearance frequency of the implicit demand feature in the subsequent interaction after the appearance of the explicit demand feature, and determining the time sequence dependence frequency of the demand extension;

Constructing a demand extension model based on the time sequence dependent frequency, wherein the demand extension model takes an explicit demand characteristic as a starting point, an implicit demand characteristic as an ending point and the time sequence dependent frequency as a weight of a side to form a demand extension relation diagram;

Identifying a target extension path starting from the explicit demand characteristics through the path analysis function of the demand extension relation diagram, wherein the implicit demand characteristics corresponding to the target extension path end points are extension directions of user demands, and the weight of the edges of the target extension path is greater than the set weight;

The extending direction is subjected to structural representation, and a requirement correlation feature comprising an extending starting point, an extending path and an extending end point is generated, wherein the structural representation is used for describing a specific path of extending the user requirement from the explicit expression to the potential requirement.

6. The method for personalized recommended interaction of a store in combination with a user representation according to claim 1, wherein the step of obtaining the current interaction environment characteristics of the store by using a scene feature extractor comprises the steps of:

Acquiring coordinate distribution data of a commodity display rack through a store electronic map system, extracting the space position of each commodity and the distance data of adjacent commodities, and generating commodity space distribution characteristics;

Acquiring online state, screen brightness and speaker volume data of an interactive terminal from a store equipment management system, and generating equipment state characteristics;

Counting the triggering times of position moving signals in a current time window, calculating the number of users entering a store in unit time and the user density data of each area, and generating time sequence flow characteristics;

Carrying out standardization processing on the distance data of the commodity space distribution characteristics, carrying out normalization processing on the brightness and volume data of the equipment state characteristics, and carrying out sliding window smoothing processing on the user density data of the time sequence flow characteristics to obtain processed commodity space distribution characteristics, equipment state characteristics and time sequence flow characteristics;

And carrying out feature fusion on the processed commodity spatial distribution features, the equipment state features and the time sequence flow features according to the preset dimension positions to generate interaction environment features.

7. The method for personalized recommended interaction of stores in combination with user portraits according to claim 6, wherein counting the number of triggers of the position movement signal in the current time window, calculating the number of users entering the store in unit time and the user density data of each area, generating the time sequence flow characteristics, comprises:

Determining the starting time and the ending time of the current time window, and extracting data of a time stamp in the time window from the position moving signal;

counting the number of position coordinate points appearing for the first time in the data, and taking the number as the number of users entering a store in unit time;

dividing a store into a plurality of regional units with equal areas, and counting the number of position coordinate points in each regional unit;

calculating the ratio of the number of the position coordinate points of each area unit to the area of the area unit, and taking the ratio as the user density data of the area;

And carrying out logarithmic transformation processing on the user quantity and the user density data of each area to generate a time sequence flow characteristic.

8. The method for personalized recommended interaction of store in combination with user portraits according to claim 1, wherein said performing feature fusion reasoning process on said user intention feature vector and said interaction environment feature through a recommendation decision network, generating personalized recommended instructions, comprises:

Inputting the user intention feature vector and the interaction environment feature into a feature fusion layer of the recommendation decision network, screening feature dimensions contributing to recommendation decisions through a gating mechanism, and generating a fusion feature vector;

the position reasoning sub-network of the recommendation decision network is utilized to conduct spatial position prediction on the fusion feature vector, the probability value of the user staying in each region is calculated, and the region with the highest probability value is selected as the triggering position;

Extracting a keyword vector of an explicit feature required in a user intention feature vector and an attribute feature vector of a commodity sold in a store through a content matching subnetwork of the recommendation decision network, calculating a cosine similarity value of the keyword vector of the user intention feature vector and the attribute feature vector of each commodity, and taking a corresponding commodity with the cosine similarity value larger than a set threshold value as a content identifier;

Selecting a presentation mode adapted to the current equipment state as a response mode based on the response adaptation sub-network of the recommendation decision network and the screen brightness and speaker volume data of the equipment state characteristics;

and carrying out consistency check on the probability value of the triggering position, the matching degree of the content identifier and the matching degree of the response mode, and carrying out instruction encapsulation on the content identifier, the triggering position and the response mode which pass the check, so as to generate the personalized recommendation instruction.

9. The method for personalized recommended interaction of a store in combination with a user representation according to claim 1, wherein executing the personalized recommended instruction and collecting a user response signal comprises:

Presenting commodity information of the content identifier at the triggering position of the personalized recommendation instruction through corresponding equipment, and triggering user interaction according to the response mode;

Collecting stay time data of a user at a trigger position through a sensor array of a position moving signal, collecting contact frequency data of the user on recommended goods through a pressure sensor of a goods contact signal, collecting feedback voice data of the user on recommended content through a microphone array of a voice interaction signal, and generating a user response signal;

Inputting the user response signals into a parameter updating module of the space-time characteristic encoder, calculating output errors of a time sequence coding layer, a space coding layer and an intention extraction layer, and adjusting weight parameters of each layer through a back propagation algorithm;

Inputting the user response signals into a weight optimization module of the recommendation decision network, analyzing the deviation value of the matching degree of the recommended content and the feedback of the user, and optimizing the inference weights of the position inference sub-network, the content matching sub-network and the response adaptation sub-network through a gradient descent algorithm;

and synchronously storing the updated space-time characteristic encoder parameters and the optimized recommended decision network weights to form an iterative optimized recommended interaction system.

10. A store personalized recommendation interaction system combined with a user portrait, which is characterized by comprising a processor and a memory, wherein the memory is connected with the processor, the memory is used for storing programs, instructions or codes, and the processor is used for executing the programs, instructions or codes in the memory so as to realize the store personalized recommendation interaction method combined with the user portrait according to any one of claims 1 to 9.