ES2818423A1 - DISPLAY DEVICE FOR COMMERCE AND PROCESSING METHOD OF IMAGES CAPTURED BY THE SAME - Google Patents

Abstract

Retail display device and image processing method captured by it. Display device for commerce that comprises broadcasting means (40) of multimedia content (40a) associated with a product (1) offered by the store, towards at least one customer (2); capture means (41) for captured images (41a) of the physical environment (3) of said store; processing means (42) of said captured images (41a) of said physical environment (3); also with behavioral recognition means (44) of at least one customer (2), configured to monitor their spatial and / or gestural behavior in the physical environment (3) of the display device, as well as for the recognition of their demographic attributes and / or their emotional response. Associated method of processing images captured (41a) by said display device, so that it captures information from its physical environment and processes it in order to optimize the information to be displayed to maximize the customer's interest in the product (1) displayed.

Description

DESCRIPTION

DISPLAY DEVICE FOR TRADE AND PROCESSING METHOD OF

IMAGES CAPTURED BY THE SAME

OBJECT OF THE INVENTION

The present patent application has as its object a display device for commerce, according to claim 1, as well as a method of processing images captured by the same according to claim 14, both incorporating notable innovations and advantages.

BACKGROUND OF THE INVENTION

At present there are a series of display devices for shops that show multimedia content of image and sound, in relation to the products that are offered on various shelves. Said multimedia content may be generic, in relation to the products on display, or present a certain degree of customization, depending on specific types of the product displayed on said shelves, and may also be based on the customer's interaction with the device, depending on the interface that includes.

In particular, it is known from the state of the art, as described in document US10580052B2, systems and methods for controlling shelf display units and for graphically presenting information on shelf display units. A system includes one or more display units configured to couple to the shelves of a modular shelving system, one or more processors communicatively coupled to the one or more display units, one or more memory components communicatively coupled to one or more processors , and a series of instructions stored in one or more memory components. When run by one or more processors, the instructions cause the system to determine a product identifier for a product, identify one or more display units to update in response to determining the product identifier for the product, and update a graph output of one or more display units identified to update in response to the identification of one or more display units to update.

It is also known from the state of the art, as described in document US7798652B2, displays in a front area of a display panel, on which an article is placed. Each of the screens transmits and projects an image from the projector of luminous flux projected from the back of the screens to the front of the screens. When the luminous flux of the projector image is projected from the back of the screens, each of the internal circumferences defines an image projection space to bring the luminous flux of the projector image to the back of the screens without obstruction. A display rack can have a projector and a computer. The projector projects the luminous flux of the projector image. The screen shelf may have a reflective mirror that conducts the image flow from the projector to the screens.

In view of the above, there is a need to present a display device that captures information from its physical environment and processes it in order to optimize the stimulation to be displayed in said smart display in real time in order to maximize the interest of potentials customers present in front of the display device in the products displayed.

DESCRIPTION OF THE INVENTION

The present invention consists of a display device for commerce, and an associated method of processing images captured by it. Said display device integrates a stimulation processing layer or level and a sensorization processing layer or level, or detection by means of sensors, so that the information presented in said smart display can be personalized in real time in order to maximize the interest of consumers in the products exhibited, additionally using the previous data obtained from the behavior of these or other customers.

Specifically, the display device for commerce of the present invention comprising means of broadcasting multimedia content associated with a product offered by the store, towards at least one customer of said store; means for capturing images captured from the physical environment of said business; means for processing said images captured from said physical environment; additionally comprising means of behavioral recognition, or behavior, of at least one customer present in said store, configured to monitor their spatial behavior and / or gesture in the physical environment of the display device, as well as for the recognition of its demographic attributes and / or its emotional response. In this way, it is possible to adapt the display device the multimedia contents presented to the customer of the store according to variables corresponding to said customer, maximizing the possibilities of capturing their interest in the product offered, and that it pleases them, deciding positively regarding their acquisition.

It should be noted that the means of broadcasting multimedia content are specifically means of projection of broadcast images, alternatively being means of broadcasting sound. Specify that by tracking is understood a tracking of their physical position, and / or their movements, and therefore of whether or not they make a stop in front of the exhibiting device, or their body movements, specifically the orientation of their head and of the look, or of the facial expression, in particular and preferably in the physical environment of the display device. Mention that demographic attributes must be understood, in a general and non-limiting way, age, gender and race. On the other hand, the means for processing said captured images optionally comprise means for selecting the captured images from among the captured images of said physical environment.

It is worth mentioning, with respect to the means for capturing images captured, that these can be unique, both for detecting an interaction with the display device, and for carrying out facial recognition, as well as presenting first means of capture, specific to detect an interaction with the display device, and on the other hand a second capture means to carry out facial recognition.

Advantageously, the display device for commerce specifically comprises means for detecting the interaction of the customer with said display device, so that the display device reacts by displaying some or other multimedia content based on the customer's instructions, personalizing the information to the concerns of the specific customer who is activating the commands of the display device.

Optionally, the means for capturing the captured images comprise means for detecting the interaction such as a first camera to detect interaction, preferably, on the other hand, having a second camera to detect and record the behavior or behavior of the customer, making a behavioral recognition, or behavior.

On the other hand, the means of behavioral recognition of at least one customer present in said store, are configured to monitor and / or track the customer's interaction with the objects arranged or product in the display device, that is, if it takes or leaves a product, and the time it is held in hand, being also and / or alternatively configured to monitor and / or track user interaction with tactile projection means integrated in the display device.

In a preferred embodiment of the invention, the media for broadcasting multimedia content of the display device is a touch screen, said touch screen being tactilely interactive by the customer, so that the customer can select the information options that are most of interest to the customer. view of the products on display.

According to another aspect of the invention, the display device can optionally comprise at least one infrared frame configured to detect the passage of some physical element through it. In this way, the display device is enabled to recognize the action of the customer to bring his hand, or other object, to the displayed product, or to the display device screen where information about it is presented. Said infrared frame is optional capable of detecting the position or coordinates, by which the infrared frame has been crossed. Alternatively, said infrared frame can be placed adjacent to a glazed surface or a screen, providing it with tactility, in the sense of having the ability to detect an approaching object until it touches the glazed surface. .

In a preferred embodiment of the invention, the means for broadcasting multimedia content and / or the means for capturing images captured and / or the means for processing said images captured, are connected to control means, so that it can be carried out. a regulated management of these resources, under parameters programmed in advance.

Note that said control means are optionally located on a remote server. On the other hand, the means of broadcasting multimedia content and / or the means of capturing images captured and / or the means of processing and / or means of control are optionally connected to a database, either locally or remotely. . The connection to the control means is preferably via a router or 4G modem establishing the connection through the internet, benefiting from the benefits and advantages of said infrastructure.

According to another aspect of the invention, the means of broadcasting multimedia content associated with a product are, optionally, at least one virtual reality representation device, so that the customer can receive information about the product in a more complete and detailed way, recreating before his eyes a physical environment different from the real one. The virtual reality representation device displays information associated with the selected product, either by means of an external touch interface, or directly by picking up the product.

Complementarily, the display device for trade comprises at least one support structure that is configured to support at least one product of said trade, so that said product is placed safely in front of the customer, the latter being able to optionally take it and return it. to leave in place.

More specifically, the display device optionally comprises at least one RFID antenna configured to establish a wireless connection with at least one RFID tag located on the product, so that the display device can detect the action of the customer taking the product, optionally triggering the specific presentation of multimedia content.

It should be noted that the RFID antenna is located on a support structure of the display device, preferably in a support area for said product. Optionally it comprises a common RFID antenna for all the products of the display device, and alternatively it comprises an individual RFID antenna for each support area of each product. With this last option of an individual RFID antenna for each support area of each product, the detection reliability of the action of taking the product is greater.

Advantageously, the display device comprises means for lighting the product in its area of support on the support structure, so that the visibility of the details of the product by the customer is greater. Optionally, the lighting means are RGB LED diodes configured to be controlled by control means, so that it is possible to select and vary the color of the light emitted on the product.

In a particular embodiment of the invention, the display device comprises a support structure in a substantially horizontal position, and media for broadcasting multimedia content in a substantially vertical position, so that the multimedia content is displayed in the most suitable orientation, according to the direction of a substantially horizontal gaze of the customer towards the product placed, usually vertically, on the support structure. Mention that the means of broadcasting multimedia content are specifically means of projection of broadcast images, preferably located behind the support structure, understood from behind as in a back plane, according to the direction of observation of the client.

In another particular embodiment of the invention, the display device comprises a support structure in a substantially horizontal position, and media for broadcasting multimedia content in a substantially horizontal position, so that the multimedia content is displayed in the most suitable orientation, according to the direction of a substantially vertical gaze from top to bottom of the client towards the placed product, on the support structure. Mention that, in this particular embodiment, the support structure can be substantially transparent, specifically a glass, and the media for broadcasting multimedia content, preferably means for projection of broadcast images, are located below the support structure. Optionally, the support structure is the surface of the broadcast image projection means itself.

In another particular embodiment of the invention, the display device comprises means of broadcasting multimedia content associated with a product, in a substantially vertical position and at least one touch screen configured for interaction with the customer, so that the customer can operate with the touch screen, influencing the multimedia content received, in relation to the product of your choice.

Optionally, said touch screen is located below the multimedia content emission means and in an inclined position, substantially perpendicular to the direction of the gaze of a customer located in front of the display device. In this way, the menu of options can be designed autonomously towards the client, and the multimedia contents presented, offering greater possibilities of interaction.

According to another aspect of the invention, the display device comprises means for collecting environmental data in the physical environment of a display device, wherein the environmental data is related to at least one variable from the group of temperature, humidity, ambient noise and / or lighting, so that it is possible to additionally personalize the multimedia content presented to the client, while at the same time it is possible to collect information from environmental data in the physical environment of the exhibiting device to process them, eventually, centrally, in front of a multiple network of devices exhibitors, thus establishing a platform for collecting environmental data in the physical store environment, with a variable number of display devices, connected to each other, for example via the internet.

Another object of the present invention is an associated method of processing images captured by a display device for shops, which comprises the steps of i) capturing images captured from the physical environment of said shops; ii) detection of a client of said business; iii) monitoring the physical position and / or movements and / or body movements and / or facial expression of said merchant customer; iv) obtaining a plurality of metrics associated with the physical position and / or the movements and / or the body movements and / or the facial expression of said merchant customer; v) determination of the degree of interest of the client in at least one product presented in the display device based on said metrics. In this way, it is possible to gauge the interest of a customer in a certain product, showing specific multimedia content associated with said product, and also to make a summary of the reaction of a multiplicity of customers, who have approached one or more devices exhibitors, or one or more shops.

In a preferred embodiment of the invention, monitoring the client's body movements and / or facial expression comprises monitoring at least one variable from the group of yaw, pitch, roll, proximity and time looking at a means for capturing captured images. In this way, it is possible to more accurately calibrate the customer's interest with respect to a certain exposed product.

More specifically, the determination of the degree of customer interest in at least one product presented in the display device is based on a linear combination of the number of stops with gaze in front of the display device, with the number of interactions with the product, and with the number of touches on a touch screen and / or with the number of interactions on a virtual reality rendering device. In this way, the quantitative value of the variable degree of customer interest is determined with high precision, by considering a plurality of factors, all of them directly related to a greater probability that the customer acquires the displayed product.

Point out that a preferred linear combination of gaze stops, 'picks', or interactions with the product, and 'clicks', or number of touches on a touch screen and / or number of interactions on a virtual reality display device, it is specifically the sum of gaze stops, plus the number of 'picks' multiplied by a factor of 1.5, plus the number of 'clicks' multiplied by a factor of 2.

It is worth mentioning that the determination of the degree of interest of the client in at least one product presented in the display device is a function of the location of the display device, the environmental and temporal conditions, calculated from the training of 'machine learning' algorithms supervised in view of the historical data of said degree of interest. In this way, the degree of customer interest is estimated based on conditions such as type of audience, location, based on what happened in the past.

Complementarily, the determination of the degree of interest of each audience group, such as a plurality of customers, in at least one product presented in the display device is a function of the location of the display device, the environmental and temporal conditions, calculated from of the training of supervised 'machine learning' algorithms in view of the historical data of this degree of interest.

Alternatively, interactions with the product can take place through a virtual reality representation device, specifically VR virtual reality glasses, which can display information associated with the selected product, either through an external touch interface or directly by taking the product.

The attached drawings show, by way of non-limiting example, a display device for commerce and a method of processing images captured by it, constituted in accordance with the invention. Other characteristics and advantages of said display device for commerce and said image processing method captured by the same, object of the present invention, will be evident from the description of a preferred, but not exclusive, embodiment, which is illustrated by way of non-limiting example in the accompanying drawings, in which:

BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1.- Schematic view of a plurality of 'Phygi' display devices connected to a remote server, equipped with artificial intelligence, according to the present invention;

Figure 2.- Schematic view of a block diagram with a display device, connected to a server and a database, including means for controlling and processing the images captured, according to the present invention;

Figure 3.- Schematic view of RFID tags in the vicinity of RFID antennas in connection with control means, according to the present invention;

Figure 4.- Perspective view of a first model of the display device, as an interactive shelf, according to the present invention;

Figure 5.- Perspective view of a second model of the display device, as an interactive shelf, according to the present invention;

Figure 6.- Perspective view of a third model of the display device, like a showcase, according to the present invention;

Figure 7.- Perspective view of a fourth model of the display device, as a virtual reality totem, according to the present invention;

Figure 8.- Perspective view of a first implementation of the first model of the display device, as an interactive shelf, according to the present invention;

Figure 9.- Perspective view of a second implementation of the first model of the display device, as an interactive shelf, according to the present invention;

Figure 10.- Perspective view of an implementation of a second model of the display device, as an interactive shelf, according to the present invention;

DESCRIPTION OF A PREFERRED EMBODIMENT

In view of the aforementioned figures and, in accordance with the numbering adopted, an example of a preferred embodiment of the invention can be seen in them, comprising the parts and elements that are indicated and described in detail below.

As can be seen in figure 1, the invention can consist of a plurality of 'Phygi' display devices, each with its physical environment (3), connected to a remote server (8), optionally in the cloud, optionally equipped artificial intelligence. The connection (82) is usually via the internet (83).

Thus, in an embodiment with a plurality of display devices, there is a common infrastructure for all, the server (8), each display unit or device being connected by 4G, or successive generations of mobile telephony, to a server (8) in the cloud, which tells you what to show at all times, and where all the information captured is sent to be analyzed.

As can be seen in the block diagram of figure 2, the invention comprises a display device of at least one product (1), connected to a server (8) and a database (81), including capture means ( 41) of captured images (41a), as well as control means (47), processing means (42), selection means (43) and behavioral recognition means (44) and interaction detection means (45), through which a series of metrics (91) of the degree of interest (92) of the client (2) with respect to the product (1) is reached, in particular as a result of the calculation of a linear combination (93) of variables detected relative to the movements and behavior of the customer (2) in the physical environment (3) of the display device.

As can be seen in figure 3, the display device of the invention comprises at least one RFID tag (73) in the vicinity of at least one RFID antenna (72) in connection with control means (47), including controllers and drivers of said RFID antenna (72).

Figure 4 shows a perspective view of a first model of the display device, as an interactive shelf, optionally including at least one light source, preferably LED, on the screen or emission means (40) from multimedia content (40a). The objective in this first model is to highlight high-end products (1) and provide increased information on these, in addition to improving the shopping experience.

Thus, it can be considered an intelligent shelf with the ability to encourage consumption, improve the customer experience (2) and obtain information on their behavior as a buyer. This embodiment includes a 43 '' capacitive screen, an RGB-D camera to detect the interaction, an RGB camera for facial recognition and a computer to process the information. It is designed to place a series of products (1), depending on their size, while multimedia content (40a) is broadcast by the broadcast means (40) or screen. In the event that the customer (2) picks up a product (1), the interaction is detected by means of a camera with a computer vision algorithm, or artificial vision, located in an overhead manner, and multimedia content is launched (40a ) particular in addition to augmented product information. It comprises a touch screen (40b) that can be interacted with by having tactility. The facial recognition camera makes it possible to detect and characterize the audience that interacts with the display device.

Preferably, the operation is controlled by an application developed in a graphics engine that connects to the management platform and displays the information of the products (1) displayed when they are picked up by customers (2), recognizing this interaction through the interaction of a RFID antenna (72) and at least one RFID tag (73). All the information on the products (1) and the multimedia content (40a) are in the database (81), and can be updated at any time in a centralized manner.

Figure 5 shows a perspective view of a second model of the display device, also in the manner of an interactive shelf, in which one or more capture means (41) per display device can be located in the upper position, on one side, or on another that is deemed convenient. It preferably comprises lighting means (48) either in the central space, where the broadcasting means (40) of multimedia content (40a) are located, although it is more common for said lighting means (48) to be located in the cells contiguous, also with products (1), but without broadcasting means (40) of multimedia content (40a), by way of substitution thereof. Optionally, it comprises an infrared frame (5) at the outer edge of the central space with the broadcasting means (40) of multimedia content (40a).

It should be mentioned that said second model of display device is preferably specified in a multiple vertical shelf, where a wide range of products (1) in a sales space can be highlighted. It also has the objective of encouraging consumption, improving the customer experience (2) and obtaining information on their behavior. It includes broadcasting means (40) of multimedia content (40a), specifically a 65 '' screen, placed at the bottom in a vertical orientation, on which the products (1) are placed in front on shelves. In front of the screen and the products (1), an infrared frame (5) is placed, optionally, and as an alternative to RFID detection technology, to detect when a customer (2) introduces their hand, or physical element (51 ), to take a product (1). This interaction produces a change in the content, giving information about the product (1) with which they have interacted and creating an experience around the purchase. In addition, and preferably, it has capture means (41), specifically a facial recognition camera to detect the audience and characterize it.

Preferably, the operation of the display device mode as multiple vertical shelving is controlled, like the embodiment of figure 4, by an application developed in a graphics engine with similar features.

Figure 6 shows a perspective view of a third model of the display device, like a showcase, which can be horizontal or vertical, although the horizontal case is shown, as a preferred embodiment. The difference with the models in figures 4 and 5 is that it uses an outer glass surface that protects the product (1) and prevents direct access by the customer (2). Said outer glass surface can be endowed with interactive features, by placing an infrared frame (5) located on its perimeter.

Thus, said model of display device, according to figure 6, can be classified as an interactive digital display case. At the bottom of the showcase, broadcast media (40) for multimedia content (40a) or a 43 '' screen is placed that improves the experience of buying the product (1). Products (1) that want to be displayed are placed on top of said screen. A clear example is expensive watches or jewelry, which is not directly accessible by the consumer or customer (2). As has been said, an infrared frame (5) is placed on the glass located in the upper part of the showcase, giving it tactility, thus making the screen interactive through the upper glass, creating an effect positive experiential towards the client (2). This model is more oriented towards luxury goods, exhibitors of fairs, events, etc. Like the previous figure models, it has a facial recognition camera to detect the audience and characterize it.

Preferably, the operation of the display device mode as multiple vertical shelving is controlled, like the embodiment of Figures 4 and 5, by an application developed in a graphics engine with similar features, connecting with the management platform and displaying the information of the products (1) displayed when they are picked up by customers (2). In this case, however, the interaction is achieved by tapping on the top of the display case, since the products cannot be picked up. All product information (1) and multimedia content (40a) can likewise be in the database (81) and can be updated centrally at any time.

Figure 7 shows a perspective view of a fourth model of the display device, like a virtual reality totem, optionally including one or more capture means (41). As an additional technological element, a virtual reality representation device (6) is included, specifically virtual reality glasses, supported on a base that enables their electrical charge by contact, at the same time that it supports them. Additionally, it has an alarm that goes off if the customer (2) separates, for example, more than 3 meters from the display device, warning, for example, previously at 2 meters.

As can be seen in figure 7, the design is a different variant from the previous ones, in the idea of arriving at a type of display device that brings closer the natural and simple use of virtual reality in public spaces, such as a store , a museum or a company facility, in a functionality as an experiential totem or as a totem for virtual reality training. A touch screen (40b) is included to select the experiences, and preferably several virtual reality representation devices (6), or VR glasses, to be used by customers (2). These VR glasses are designed to show images or video, optionally at 360 °, and / or showing 3D scenarios

As in the models in the previous figures, the functionality is managed by an application developed in a graphics engine that connects to the platform of management, optionally located remotely in a server (8) and in a database (81), establishing communication through a connection (82) via the internet (83).

The information of the multimedia contents (40a) and virtual reality are uploaded through said platform, such as tourist videos depending on the place, images of cars or any other product (1) or service that wants to be exposed. Similarly, all the information is in the database (81) and can be updated centrally at any time.

It should be noted that said totem-type display device model preferably has an upper screen, as media for broadcasting (40) multimedia content (40a), and a lower touch screen intended for interaction with the customer (2), which shows the products (1) and / or experiences to be visualized. Specifically, the experiences must be dragged to the representative icon of each VR glasses, leaving said VR glasses ready to view the experience.

In figure 8 it can be seen, a first example of implementation of the first model of the display device, as an interactive shelf, specifically presented wine products (1).

In figure 9 it can be seen, a second example of implementation of the first model of the display device, as an interactive shelf, specifically presenting sports footwear products (1), with a customer (2) taking a product (1), while the capture means (41) of captured images (41a) corresponding to a customer (1) perform their function, thus making it possible to improve the shopping experience.

In figure 10 you can see an example of implementation of the second model of the display device, as an interactive shelf, optionally without support structures (7), specifically presenting interior design products (1), with a client (2) taking a product (1), while the capture means (41) of captured images (41a) corresponding to a customer (1) perform their function, thus making it possible to improve the shopping experience.

In relation to other more detailed aspects of the present invention, it should be mentioned that said display device integrates a layer or level of sensorization processing, or detection by means of sensors, aimed firstly at the control of the audience, by means of computer vision, and secondly at the control of the interaction, by means of RFID identification and interactions with the displays of the display device.

In the control of the audience, from the computer vision a series of metrics (91) are obtained for each client user (2). The system comprises several serial processes: detection, tracking or follow-up, and obtaining metrics (91). Detection is preferably carried out with a library called OpenVINO, developed by Intel Corporation. Mention that facial detection models will be used, for example those related to gender, age and / or emotional state. Regarding tracking, a unique and consistent identifier is assigned to each client user (2). The temporary processing of the different variables obtained previously or metrics (91), are associated with a subject identified as a client user (2), for their follow-up. The tracking algorithms are built sequentially in order not to lose track of the subject identified as client user (2).

Regarding the obtaining and processing of metrics (91), as variables corresponding to each subject of the audience, or group of client users (2) in a certain physical environment (3), they preferably comprise a) three different angles of facial orientation , such as yaw, pitch and roll); b) proximity to the display device; and c) if the client user (2) has been looking at the camera. The metrics (91) per client user (2) are recorded in a database (81.)

In relation to the architecture of sensorization, detection is carried out with a computer vision algorithm, whose input or input is an RGB camera, or capture means (41), and whose output or output is called Raw Data 1, or raw data 1, as shown in figure 2. Next, and in terms of tracking, the Python Script 1, or SmartShelf algorithm of the exhibiting device generates Raw Data 2, or raw data 2, from Raw Data 1. Subsequently, the obtaining and processing of metrics takes place (91). Python Script 2, or Server algorithm, present on the server (8), processes the uploaded data and generates a set of metrics (91) processed and classified. Additionally, it uploads the processed data to the database (81), preferably of the SQL type.

It is worth mentioning that to measure consumer or customer behavior (2), a layer of analysis of the audience of the display device, or smart shelf, has been developed using computer vision. This has been developed to analyze videos in real time, although it allows its application in post-processing to analyze videos already captured in order to diagnose possible errors and improve the solution. The analysis consists of obtaining different metrics (91) for each user or client (2) that has appeared in the video in a specific space of time. As has been pointed out, the developed system has serial processes, specifically those of detection, tracking or monitoring and obtaining metrics.

More specifically regarding the detection process, different models of Machine Learning are used, or automatic learning, specifically Deep Learning, among others those that can perform car detection and automatic license plate reading, people detection or face detection. These models work in real time, both for a video and for an image. In this case, facial detection models will be used, such as the detection of age and gender, the mood of a person or the orientation of their head. Thus, a series of measures related to gender, age, emotional state (happy, neutral, sad, angry or surprised) and the orientation of the head of each subject or client are obtained (2). A model has also been incorporated which detects the size and position of a human face, on the studied video, according to a square.

It should be noted that the facial detection models analyze each image independently, so they cannot track people or customers (2). By detecting faces at a given moment, but without an identifier allowing the same face to be identified between frames, we can know how many faces (and by extension people or customers (2) there are at any given moment, but not how many different faces have passed during a period of Therefore, it is necessary to implement an algorithm that performs tracking, that is, that allows assigning a unique and consistent identifier to people or customers (2) over time.

From the above, a data sequence of the different detections is obtained in real time. The detection frequency is approximately 25Hz-20Hz, the variation depends on the number of subjects or clients (2) in the scene or physical environment (3). Thus, the more subjects or clients (2), the more processing and therefore the frequency decreases. The detections are saved in real time in a table format. For the videos to analyze are not very long, continuous capture from the camera or capture means (41) is segmented into sessions, which are defined from the moment a person or client (2) enters the scene until no face is detected for 1 second . When this happens, the detection during this session is exported, preferably in Excel format, for further processing. In order to avoid that the processing of a video or session is very long, it is stipulated that after three minutes of uninterrupted detection, the detection is cut by saving the data of the three-minute session and restarting the detection algorithm automatically. Said data is stored in a temporary format where each data is assigned the time in which it was obtained without it being consistent with the subject or client (2) on which a specific measurement has been made. For this, the second processing algorithm is used: tracking.

As for the second algorithm or tracking algorithm, it deals with the temporal processing of the different variables obtained previously, associating them with a subject or client (2) identified with a unique integer in the session. To do this, a series of concatenated algorithms for tracking the subject or client (2) are used, in order to associate the variables obtained from the first algorithm with the subjects or clients (2) in the scene or physical environment (3). These algorithms basically study the evolution of the position of the subjects as they move. A unique identifier is associated with each initial subject throughout the session. By comparing two temporally followed data sets, the initial identifiers are associated with the closest ones in the next data set. This association of identifiers is not only done through proximity algorithms; Others are used that take into account, for example, the speed of the subjects or clients (2), and the optical flow.

As mentioned, these tracking algorithms are built sequentially, that is, first the data goes through a first algorithm, these results are processed by a second algorithm, etc. Regression machine learning models have also been implemented that base their knowledge on the movement of the subject or client (2), during the session and then predict their position a certain time later in order not to lose track of this person. This allows, for example, that if a subject is not detected for a certain time in the scene and then it is detected again, its tracking is not lost. That is, the future position of the subject or client (2) is interpolated to continue with their monitoring. In addition, it is also explicitly prevented that two different subjects may have the same identifier, for example as a consequence of two subjects or clients (2) crossing each other during a long time interval, which may cause the algorithm to assign the same identifier to two subjects or clients (2). For this, the subjects or clients (2) with the same identifier are reviewed, assigning the repeated identifier to that subject or client (2) who has had a trajectory as similar as possible to the prediction made.

Next, we move on to the third processing phase, calculating a series of metrics (91) that analyze the audience throughout the session, thus obtaining the average age, gender and average emotional state, among other variables, of each subject. or customer (2). Analyzes are also carried out that count the number of times that a subject or client (2) has stopped, has positioned himself to the left or right according to the video reference system or has approached within a distance of two meters .

To obtain these metrics (91), more detailed previous studies have been carried out on the videos already recorded or on real-time sessions where a series of parameters are studied that mark limits between, for example, near or far, or if the subject or client (2) looks or does not look within a certain range. For this, the data of the orientation of the subject or client's head (2) is processed, which marks us three different angles called yaw, pitch and roll. Said angles are calibrated by studying different videos and with this a threshold is marked for when a subject or client (2) is looking within a certain spatial range as an area of interest in the physical environment (3). The same is done for the distance at which the subject or client is (2). The camera or capture means (41) that achieves the recording of videos, does not have a depth component, which gives us data that is found on a really flat image, without a sense of depth. Despite this, it is possible to establish a parameter that marks the proximity of a subject or client (2) with respect to the camera or capture means (41), thus segmenting the position of the subject or client (2) into near and far.

Regarding the measurements detected on the faces of the subjects or clients (2), an additional variable can be established that assesses whether a person has been looking for a certain time within a viewing angle during the session. If you have been doing it, the metrics (91) related to this subject or customer (2) are obtained during the viewing time, eliminating the records where you have not been looking. Thus, the information derived from these subjects or clients (2) will be obtained only from instants of time where said subject or client (2) has been looking at the camera or capture media (41), and therefore, those variables are much more robust than those that can be obtained by averaging over the entire session. More complex metrics (91) have also been studied, such as the sequence of movements of a subject or client (2). This sequence marks the order of actions carried out by said subject or client (2), differentiating between approaching, moving away, left and right, and looking or not looking. This allows the subjects or clients (2) of the same session to be grouped according to movement patterns.

The set of variables described above is grouped and summarized in a table that is automatically saved in a database (81), registering as an analysis by subject or client (2) of a session. A third data processor, within the same algorithm, performs the analysis per session, thus reporting, for example, the number of men and women who have participated in it.

Alternatively, it is established that the various algorithms work in parallel so that the data collection of the first algorithm and the data processing by the second algorithm of already saved data are achieved at the same time. Since the second processing algorithm normally takes more time than the first, all the data to be processed are saved and recorded in a history in the database (81).

Finally, the data belonging to the metrics (91) per user or client (2) are recorded in the database (81) in order to be able to graphically display the results achieved, and generate reports. Furthermore, these algorithms are a single program that can run on a single computer core, making it possible for the algorithm to run on as many cores as the computer has, doubling or quadrupling its speed.

In relation to the control of customer interaction (2) with the display device, interactions with touch screens (40b) are considered first, by means of light keystrokes or touches, relating it to each product (1), and giving rise to the generation of metrics (91), for example with the number of touches. Secondly, radio frequency or RFID identification technology is considered, optionally with RFID HF sensors, and alternatively with RFID UHF sensors, integrated in the display device: The products (1) comprise an RFID tag or labels (73) that make It is possible to know through the sensor if a tag is removed from the accessible field of the RFID antenna (72) or if it is returned to the initial position.

Regarding the architecture related to the control of user or client interaction (2), it preferably includes an antenna controller, optionally of the type (XN), connected to control means (47) or control unit (Raspberry), which is the device that stores the algorithm to translate the signals and send the corresponding events to the computer that stores the application in charge of displaying the actions on the screen or broadcast media (40). On the other hand, it has tags on the products, or RFID tag (73), and RFID antennas (72), with a specific driver or controller, connected to the control means (47) or control unit (Raspberry). In turn, it is preferably connected to a 4G router through which the container computer in charge of launching the application is connected.

The screens used to capture the interaction with the user or customer (2) are tactile, in order to receive inputs based on the interaction of each user or customer (2) with the touch screen (40b), relating it to each product ( 1). All this is collected in the application, generating metrics (91) representative of the degree of interest, such as the number of times each information tab relative to each product is clicked or pressed (1).

Note that, preferably, there must be a minimum distance between RFID antennas (72) to avoid interference and erroneous detections. This minimum distance must be 8.5 cm horizontally and 6 cm vertically so as not to cause interference with the metal that is part of the display device.

It should also be mentioned that each RFID antenna (72) has a specific detection range, therefore, when installing the components, it is necessary to consider that the distance from the RFID tag or label (73) to the RFID antenna (72) is within its detection range. Specifically, the maximum distance is 2.2 cm.

On the other hand, the RFID antenna controllers (72) are directly connected to the control means (47) which is the device that stores the algorithm to translate the signals and send the corresponding events to the computer that stores the application in charge of displaying the shares by the means of issuance (40) or screen. The sending system is carried out through MQTT (Message Queing Telemetry Transport, or Message Queue Telemetry Transport), which is an M2M (machine to machine). This protocol is based on a TCP / IP stack and each connection is kept open.

Said system is connected to a 4G router by means of which the container computer in charge of launching the application that will interpret the MQTT messages received by the control means (47) is connected to use them. In this case, the application is developed in a graphics engine, which will receive messages through MQTT as the interactions are recorded and will translate them into different situations on the broadcast media (40) or screen.

In relation to additional aspects of the present invention, specifically the representation and visualization of metrics (91), and virtual tables that show data of interest, it should be mentioned that, in order to extract information of interest concerning the representation and visualization of metrics (91), different views have been developed in SQL Server. These are virtual tables that show data from one or more tables and whose content is defined by an SQL query. In this way, the necessary data are synthesized and unified to obtain a complete and coherent perspective of the data set that is to be shown to the user or client (2), for those tables in which the desired information cannot be directly extracted from the structure. with which it has been stored. The objective is to avoid, as far as possible, carrying out this data consolidation within a tool that is used to visualize the information, preferably an advanced data visualization service, in order to maximize performance in the execution of said application. .

To visually represent fragments of information and data of interest, a tool has been used that is used to visualize the information, allowing the use of different data sources, and displaying them graphically in an interactive way. The flow for the creation of reports and their subsequent online visualization, goes through the use of a desktop application for the advanced data visualization service, where a) connections with the data source are established, and b) the report is created , allowing end users or customers (2) to access and interact with the information online.

Basically, once the report is generated in an advanced data visualization service, it is published within a workspace, where reports, dashboards, data sets, etc. can be shared. with the end users or clients (2), through the connection with SQL Server.

Regarding the data connection mechanisms, the import or loading of the entire data model in memory is implemented, as it offers greater flexibility, both in data manipulation and in the implementation of calculations. A higher performance is obtained since when performing an update of the data, these are stored in the memory of the capacity of the advanced data visualization service, allowing queries to be carried out more quickly; and finally, a single data source is used. The set of represented data is not updated in real time, although multiple updates can be scheduled to run throughout the day.

Regarding the design and content of the report, such as the data set to be represented, associated with the information of a specific campaign, three different blocks can be distinguished: a) a filter panel applicable to the current tab. This area allows segmenting the displayed information, limiting the set of data that is represented in the viewing area; b) an area for displaying the information extracted from a data set, which may contain one or more visual objects. And c) a corporate logo.

Advantageously, the report comprises different tabs, the main ones being the following:

I) 'Location' tab: Shows the main indicators, as well as the geolocation of the different exhibiting devices. It visually includes different informative objects such as i) the date, allowing to limit the data displayed within a range of time; ii) the location, limiting the set of data represented according to the location of the exhibiting devices that are part of the commercial campaign, presenting multiple information fields, such as autonomous community, province, municipality, establishment or surface where the device is located ; iii) display of metrics (91), as indicators related to the efficiency and productivity of the display device; iv) location map, showing the geolocation of the establishment where the different display devices are located. A graphic resource is the size of the circle, which specifies its geographical location within the map, and represents the average daily potential audience, that is, the average number of daily visits in the selected time range; v) metric comparators (91) in the form of tables, allowing to obtain a vision of the status of the current day in comparison with the previous results.

II) ‘Traffic’ tab: Shows the volume of traffic, as well as other characteristics, that can be extracted from the audience analysis. Visually includes different informational objects such as i) Date; ii) Location; iii) Gender; iv) Age, filtering the information represented in the metric display area (91) according to the age range of the subjects, preferably <18; 19-30; 31-45; 46-55; 56-65; or> 65; v) Distance; vi) Purchase opportunities, graphically showing the behavior of potential buyers or clients (2). Represents the percentage of losses of potential customers (2) who show interest and who finally meet the business objective, that is, they look and stop; vii) Averages of purchase opportunities, representing, numerically, the data shown in the purchase opportunity funnel, and also, adding information on the average duration per visit at the different points of the funnel; viii) Total daily traffic, with the daily evolution of the total volume of potential customers (2); ix) Traffic based on distance, with the percentage of visits that pass near or far from the exhibiting device.

III) 'Flow' tab: Shows a vision of the total volume of customer visits (2), through the different behaviors mentioned above of visit - visit and look - visit, look and stop, according to a time segmentation, according to day of the week, as well as gender. Visually includes different informational objects such as i) Average daily hourly traffic; ii) Average daily traffic by day of the week; ii) Trafficking based on gender.

IV) ‘Buyer profile’ tab: Shows information on the demographic characteristics of potential buyers or clients (2) allowing to specify objectives. Visually includes different informational objects such as i) Age; ii) Gender; iii) Population pyramid, allowing a joint analysis of the demographic characteristics of both gender and age.

V) 'Emotions' tab: Shows, in addition to demographic traits, the emotional response. It establishes, based on the probability that one of the registered emotions occurs, the percentage of time in which a buyer has mostly experienced an emotion of the type of happiness, surprise, sadness, anger or neutral ... Visually includes different informational objects such as i) Emotional response, presenting the percentage of time in which an emotional response has been produced, either positive or negative, with respect to a neutral answer; ii) Type of emotional response, indicating as a percentage, the total duration in which one of the non-neutral emotions has been registered; iii) Emotional response according to the day of the week.

VI) ‘TouchEnPantalla’ tab, or touch on screen: Shows a perspective on interactivity on the touch screen (40b) of customers (2) with the display device. Visually includes different informational objects such as i) Dates; ii) Location; iii) Product (1), limiting the set of data represented according to the product (1) to which the touch is associated or touch on the touch screen (40b), allowing a multiple selection; iv) Type of display device, not allowing multiple selection; v) Daily touches according to time, vi) Touches on the different tabs; vii) Flow through the different tabs, allowing a quick reading of how customers (2) have moved through the different tabs.

VII) ‘Products’ tab: Shows an overview of the main indicators for the different display devices and products (1) that make up the campaign. Visually includes different informational objects such as i) Dates; ii) Location; iii) Category, limiting the data set represented by product categories (1); iv) Product; v) Summary of indicators; vi) Breakdown of indicators by display device and product (1), according to daily interactions, average duration of interaction, and number of daily touches.

VIII) ‘Content’ tab: Shows information regarding the reproduction of different visual audiovisual content on the display devices. Visually includes different informative objects such as i) Summary of indicators with the most significant metrics related to the content reproduced, specifically the potential audience per hour, the attraction rate per hour, the interactions per hour or the time of reproduction per day.

According to another aspect of the invention, specifically the access of the end users or clients (2) to the reports, the requirement that the end users or clients (2) have access only to the information related to the campaigns they own, including a mechanism called row-level security, consists of the creation of roles that have associated rules and that restrict user access end users or clients (2) to fragments of the data sets displayed in dashboards, reports, icons, etc. to which they have access.

This objective of limiting access to reports for end users or clients (2) is achieved through the Insert feature. This procedure requires that end users or customers (2) have a license for an advanced data visualization service, as mentioned above, and that the report has been shared with them. It is a reading view, so it cannot be modified by the consumers with whom it has been shared. Row-level security will be used to limit customer access (2) to the data they own, and to display information related to a single campaign in the report, filters will be used on the URL that is displayed. used as the content source of the iFrame or HTML element.

In order that each display device can adapt the stimulation to the sales space, the product (1) and the audience in front of it, a platform has been designed to manage the experience. This allows the design of campaigns, within which there are associated products () 1, and a series of templates on which to display multimedia content (40a).

The audience data, that is, the interactions through the RFID system and the clicks on the touch screen (40b) are sent to a server (8) to be managed. Specifically, you can use the IIS (Internet Information Services) server hosted by one of the layers offered by the AWS cloud infrastructure (Amazon Web Services) and specifically in this case, the Elastic Beanstalk layer. On the other hand, there is a relational database (81) for use in applications, specifically a SQL Server 2016 database. Said database (81) is the one that stores all the information records and through which the information of the inputs or inputs and outputs or outputs is exchanged. In turn, it contains tables distributed in different areas that are interrelated according to the purpose and nature of the data that are housed in them.

With regard to the visual functionalities presented in the display device, it should be noted, as general criteria, that the information to be displayed will depend on the position and type of product (1), following some guidelines for managing RFID tags (73 ). The information will disappear when it is returned to the RFID antenna (72) where it was, or when a defined time has elapsed, which can be 5 minutes per ex. The information to be shown in the meantime would be that corresponding to the last product lifted, that is, separated from its support position, in the vicinity of the RFID antenna (72). When all the products (1) are left in their initial position, the pattern may be to return to the rest state. The multimedia contents (40a) to be shown can be based on the audience through modulable rules that allow segmenting said multimedia contents (40a).

The general functionalities with which the display devices can be supplied, once they are connected to the management platform, can be, but are not limited to: i) Connection with the platform to recognize the templates and multimedia content (40a) to be displayed. The database (81) provides the information necessary for the operation of the display device, including modes, a list of associated reproductions, products (1) and the associated audiovisual multimedia contents (40a); ii) RFID tag management (73), including in the defined products (1) the RFID tag number (73) associated with the product (1); iii) Management of the list of associated reproductions, adapting the background to what is defined in the platform, with all the dynamic audiovisual multimedia content (40a).

Regarding what would be the idle or initial state, when all the products (1) are in position, the following audiovisual multimedia contents (40a) are displayed: The brand logo, eg. top left of the screen, languages with flags icon for language selection, title, subtitle. During this state, the videos associated with the chosen playlist contained in the multimedia content template (40a) appear in the background.

Attending specifically to the functionalities by product (1), a series of visual functionalities dependent on the interaction of consumers or customers (2) with the products (1) are included. Thus, regarding the management of the interaction of picking up or leaving the product (1), the following information derived from the RFID antenna system (72) vs RFID tag (73) is received by MQTT: Product (1) with which has interacted, if it has been caught or left and the position in which it has been interacted, eg. 2§ of 4 starting from the left on the first shelf of the display unit. Once the product (1) taken has been specified, the information related to it is displayed, depending on the position of the product (1) that has been taken. The information will disappear if it is left on the RFID antenna (72) again, or a defined time elapses, for example. of 5 min. Information to be displayed is defined by templates, which contain tabs, and each one has a design and information on multimedia content (40a). In the event that several products (1) are lifted, various information would be displayed, corresponding to each of the products (1), sequentially, ending with the information of the last product (1) lifted. And if all the products (1) are left, in their position the rest state is returned.

Optionally, the visual functionalities can be intelligent, that is, based on the particular audience at all times. Thus, and in order to optimize the multimedia contents (40a) to be shown depending on the audience, a connection of the audience metrics (91) with the broadcast media (40) has been made. In this sense, a series of modulable rules have been developed that allow segmenting multimedia content (40a) based on the estimated age and gender of the consumers or customers (2) who are in front of the display device. In this way, this last layer or level of rule-based processing provides the display device with an intelligence that allows adapting the multimedia content (40a) stored on the platform for a specific brand or product (1) depending on the audience it has. in every moment.

In addition to the above, point out the possibility of integrating Artificial Intelligence (AI) models in the management of the functionalities and the broadcasting of multimedia content (40a) of the exhibiting device. A first typology of Artificial Intelligence models would be the one that would establish a generic priority, depending on the day and time, for the existing multimedia content (40th). A second type of Artificial Intelligence models would be applied in real time, recognizing the dynamics of the audience at all times. In this regard, the concept of the interest rate, or Interest score, is relevant, calculated as a linear combination of stops with gaze, 'picks', or interactions with the product, and 'clicks', or number of touches.

Specify that supervised machine learning algorithms are used to create artificial intelligence models. Specifically for a first artificial intelligence model without the audience, establishing a generic priority based on the day and time, the objective is to predict which multimedia content (40a) is executed when no client (2) interacts, given a range of time of a specific day. The predictive model, given prior information on the operation of previous multimedia content (40a) in a specific time frame, learns which ones have been the ones that maximize interest. This model can be run several times a day to change the probability of each of the multimedia content (40a) available to be shown, thus broadcasting more times those that maximize interest.

An example of use is cited: On a Tuesday in February at 4:00 p.m. the model is run to find out which multimedia contents (40th) are shown to be shown during the afternoon slot that day. The parameters for that afternoon are entered into the model: Tuesday in February, from 5:00 p.m. to 9:00 p.m., a cold, humid and rainy day, in a given city. With this information, the model analyzes how much interest each multimedia content will create (40a), and it will give a greater probability to those with the greatest interest that they will create, for example: multimedia content (40a) with a warm environment and with female actresses, as it is the majority public women.

The main inputs or inputs can be considered those of i) multimedia content (40a), with the identifier data, keywords, authors, influencers, etc .; ii) location, with city, region and country data; iii) Environmental information, with data on temperature, humidity and rainfall); iv) temporary, with the data of time slot, day of the week, month. The value of the interest rate, or Interest score, can be issued as output or Output, calculated according to the formula £ (# ParadasconMirada + 1.5 * picks + 2 * clicks]

With respect to a second typology of Artificial Intelligence models, with a selection of specific multimedia content (40a) for an audience, although being applied in real time, recognizing the dynamics of the audience at all times. Its objective is to predict which multimedia content (40a) to execute when no client (2) and that maximizes interest with the display device, given an audience at a specific time.

The predictive model takes into consideration the previous information on the operation of previous multimedia contents (40a) with a specific audience, learning which ones have been those that maximize interest. This model is executed several times a day to change the probability of each of the available multimedia contents (40a) to be shown, depending on the audience that the display device may have in front of, for a given time slot. Once the results of the model have been pre-calculated, the probability of displaying a multimedia content (40a) for a specific audience will change each time that audience standing in front of the display device is recognized, changing the multimedia content (40a) shown to adapt them to the audience.

An example of use is cited: On a Tuesday in February at 4:00 p.m. the model is run to find out what to show when the audience changes that afternoon. In this way, the different probabilities associated with audience combinations are calculated (young men, middle-aged men, older men, young women, middle-aged women, older women, young mixed gender, middle-aged mixed gender and older mixed gender), and also taking into account the parameters of that afternoon (Tuesday in February, from 5:00 p.m. to 9:00 p.m., a cold, humid and rainy day, in a given city). In this way, the models do not have to be calculated at all times, but the results are preloaded. At a given moment, the display device detects that there are 3 middle-aged men standing in front of it, so it changes the probability of multimedia contents (40a) to those that maximize the interest rate for that audience segment that afternoon.

The main inputs or inputs can be considered those of i) multimedia content (40a), with the identifier data, keywords, authors, influencers, etc .; ii) location, with city, region and country data; iii) Environmental information, with data on temperature, humidity and rainfall); iv) temporal, with the data of time slot, day of the week, month; v) type audience. The value of the interest rate can be emitted as output or Output, calculated according to the formula £ (# ParadasconMirada + 1.5 * picks + 2 * clicks]

With respect to a third typology of Artificial Intelligence models, with a selection of increased information of a specific product (1) for clicks or pulsations of interaction. The objective is to predict the type of information to be displayed for each product (1) when interacting, which will maximize interest with said product (1) given a time frame of a specific day.

The predictive model, given prior information on the operation of content variants for the same product (1) that is shown to the client user (2) and that is selectable by him, learns which ones have received the most interest from the subject customer (2). This model is executed with the different variants of multimedia content (40a), and several times a day to change the probability of each of the available multimedia content (40a) being shown, depending on the audience that the display device may have in front of and a time slot. Once the model results are pre-calculated, the probability of displaying a variant type of product information (1) for a specific audience will change each time that stopped audience is recognized. in front of the display device, changing the multimedia contents (40a) that are displayed to adapt them to the audience.

An example of use is cited: On a Tuesday in February at 4:00 p.m. the model is run to find out what to show on the product (1) displayed on the display device when the audience changes that afternoon. For each product (1) there are 5 possible information tabs to show, of which only 3 are shown at a time. The display device has been learning for a long time by showing different combinations of 3 of the 5 possible tabs in each space (location) in each time slot and associating results with the audience profile, quantifying which tabs have been the most chosen by customers (2). In this way, the different probabilities associated with audience combinations are calculated (young men, middle-aged men, older men, young women, middle-aged women, older women, young mixed gender, middle-aged mixed gender and older mixed gender), and also taking into account the parameters of that afternoon (Tuesday in February, from 5:00 p.m. to 9:00 p.m., cold, humid and rainy day, in a specific city). In this way, the models do not have to be calculated at all times, but the results are preloaded. At a given moment, the display device detects that there are 3 middle-aged men standing in front of said display device, so the probability of the contents associated with the product (1) changes to those that maximize the interest rate for that range of audience that afternoon.

In an additional example of use: In the same way as the previous case, the information tab associated with the product (1) is only one and the variants are produced in an emitted image (40c) that is shown of the product (1). In this case, there are 10 different emitted images (40c) to show and the model will learn which of the emitted images (40c) is the ideal one based on the interest that in this case could be measured by the positive / negative emotional facial response of the client user (2) that interacts with the product (1).

The main inputs or inputs can be considered those of i) multimedia content (40a), with the identifier data, keywords; ii) location, with city, region and country data; iii) Environmental information, with data on temperature, humidity and rainfall); iv) temporal, with the data of time slot, day of the week, month; v) type audience. The value of the interest rate can be emitted as output or Output, calculated according to the formula £ (# ParadasconMirada + 1.5 * picks + 2 * clicks]

More specifically, as can be seen in Figure 2, the display device for commerce comprising means of broadcasting (40) of multimedia content (40a) associated with a product (1) offered by the merchant, towards at least one customer (2) of said trade; capture means (41) for captured images (41a) of the physical environment (3) of said store; processing means (42) of said captured images (41a) of said physical environment (3); additionally comprising behavioral recognition means (44) of at least one customer (2) present in said store, configured to monitor their spatial and / or gestural behavior in the physical environment (3) of the display device, as well as for recognition their demographic attributes and / or their emotional response.

On the other hand, as seen in Figures 9 and 10, the display device for commerce comprises means for detecting the interaction (45) of the customer (2) with the display device.

More in detail, as seen in Figure 10, the media for broadcasting (40) multimedia content (40a) is a touch screen (40b).

According to another aspect of the invention, as seen in Figure 6, the display device optionally comprises at least one infrared frame (5) configured to detect the passage of some physical element (51) through it. As an example of a physical element (51), mention the hand of a client (2).

In a preferred embodiment of the invention, as seen in Figure 3, the broadcast media (40) of multimedia content (40a) and / or the capture means (41) of captured images (41a) and / or the media processing (42) of said captured images (41a), are connected to control means (47). Additionally, the display device is connected to the internet through a router.

Optionally, as seen in Figure 7, the broadcasting means (40) of multimedia content (40a) associated with a product (1) are at least one virtual reality representation device (6).

According to another aspect of the invention, as can be seen in Figures 8, 9 and 10, the display device comprises at least one support structure (7) configured to support at least one product (1) of said trade.

On the other hand, as seen in Figure 3, the display device optionally comprises at least one RFID antenna (72) configured to establish a wireless connection (82) with at least one RFID tag (73) located on the product (1) .

Complementarily, as can be seen in figure 4, the display device comprises lighting means (48) of the product (1) in its support area (71) on the support structure (7).

In a particular embodiment of the invention, as an interactive shelf, as can be seen in Figures 4 and 5, the display device for commerce comprises a support structure (7) in a substantially horizontal position, and emission means (40) of multimedia contents (40a) in a substantially vertical position.

In another particular embodiment of the invention, as an interactive showcase, as can be seen in Figure 6, the display device for commerce comprises a support structure (7) in a substantially horizontal position, and broadcasting means (40) of multimedia content (40a) in a substantially horizontal position.

In another particular embodiment of the invention, as an interactive totem, as can be seen in figure 10, the display device for commerce comprises broadcasting means (40) of multimedia content (40a) associated with a product (1), in position substantially vertical and at least one touch screen (40b) configured for interaction with the customer (2).

According to another aspect of the invention, as seen in Figure 5, the display device for commerce comprises means for collecting environmental data (49a) in the physical environment (3) of a display device, wherein the Environmental data (49a) are related to at least one variable from the group of temperature, humidity, ambient noise and / or lighting. Its objective is to create a platform for collecting environmental data in the physical environment (3) of the store, which contemplates the characteristics of being scalable, modular, and distributed. It must be scalable, since the initial prototype can consist of a variable number of devices, data collectors and receivers. It has to be modular, since the The task of collecting data in one location can be carried out in a single device or node, or separated into several according to the conditions of the physical environment (3). It has to be distributed, since, in order to meet the above objectives, the system will consist of a platform connected through the network, with no other requirement than internet access. The part of the data receiving platform must have the ability to record and send the data to a remote server (8) or database (81) for subsequent analysis and obtaining metrics (91). The basic approach is to obtain readings for temperature, humidity, ambient noise, and lighting.

It is also an object of the present invention, as seen in figure 2, a method of processing images captured (41a) by a display device for commerce, which comprises the steps of i) capturing images captured (41 a) of the environment physical (3) of said trade; ii) detection of a customer (2) of said business; iii) monitoring the physical position and / or movements and / or body movements and / or facial expression of said customer (2) of the store; iv) obtaining a plurality of metrics (91) associated with the physical position and / or the movements and / or the body movements and / or the facial expression of said customer (2) of the store; v) determination of the degree of interest (92) of the customer (2) in at least one product (1) presented in the display device based on said metrics (91).

Specifically, and with respect to monitoring the client (2), their physical position, their movements are tracked and, therefore, whether they make at least one stop, the orientation of their head and therefore their gaze. , of the interaction of said client user (2) with the products (1) placed in the display device, and therefore of whether he picks up or leaves a product (1) and the time he has it in his hand. Likewise, tracking of the interaction of the client user (2) with the emission means (40) located in the display device, and of the interaction with a virtual reality representation device (6), specifically virtual reality glasses of the type Head mounted display, or screen mounted on the head, located in the display device.

Additionally, as seen in Figure 9, the captured image processing method (41a) comprises monitoring the body movements and / or facial expression of the client (2) and comprises monitoring at least one variable from the group of yaw, pitch, roll, proximity and time looking at a capture means (41) of captured images (41a).

Complementarily, as seen in Figure 10, said captured image processing method (41a) comprises determining the degree of interest (92) of the customer (2) in at least one product (1) presented in the display device is in based on a linear combination (93) of the number of stops with gaze in front of the display device, with the number of interactions with the product (1), and with the number of touches on a touch screen (40b) and / or with the number of interactions in a virtual reality representation device (6).

The details, shapes, dimensions and other accessory elements, as well as the components used in the implementation of the display device for commerce and the image processing method captured by it, may be conveniently replaced by others that are technically equivalent, and not they depart from the essentiality of the invention or the scope defined by the claims that follow the following list.

Numerical references list:

1 product

2 customer

3 physical environment

40 broadcast media

40a multimedia content

40b touch screen

40c images issued

41 recruitment media

41st images captured

42 processing media

43 means of selection

44 means of behavioral recognition 45 means of interaction detection

46 projection media

46th surface

47 means of control

48 lighting means

49 collection means

49a environmental data

5 infrared frame

51 physical element

6 virtual reality rendering device 7 support structure

71 support zone

72 RFID antenna

73 RFID tag

8 server

81 database

82 connection

83 internet

91 metric

92 degree of interest

93 linear combination

Claims (1)

1- Display device for commerce that includes: - broadcasting means (40) of multimedia content (40a) associated with a product (1) offered by the store, towards at least one customer (2) of said store; - capture means (41) for captured images (41a) of the physical environment (3) of said store; - processing means (42) of said captured images (41a) of said physical environment (3); characterized in that it comprises behavioral recognition means (44) of at least one customer (2) present in said store, configured to monitor their spatial and / or gestural behavior in the physical environment (3) of the display device, as well as to recognition of their demographic attributes and / or their emotional response. 2- Retail display device according to claim 1, characterized in that it comprises means for detecting the interaction (45) of the customer (2) with the display device. 3- Display device for commerce according to claim 1, characterized in that the broadcasting means (40) of multimedia content (40a) are a touch screen (40b). 4- Retail display device according to any of the preceding claims, characterized in that it comprises at least one infrared frame (5) configured to detect the passage of some physical element (51) through it. 5- Display device for commerce according to any of the preceding claims, characterized in that the means of broadcasting (40) of multimedia content (40a) and / or the means of capturing (41) of captured images (41a) and / or the media processing (42) of said captured images (41a), are connected to control means (47). 6- Display device for commerce according to any of the preceding claims, characterized in that the broadcasting means (40) of multimedia content (40a) associated with a product (1) are at least one virtual reality representation device (6). 7- Display device for trade according to any of the preceding claims, characterized in that at least one support structure (7) is configured to support at least one product (1) of said trade. 8. Display device for commerce according to claim 7, characterized in that it comprises at least one RFID antenna (72) configured to establish a wireless connection (82) with at least one RFID tag (73) located on the product (1). 9. Retail display device according to claims 7 or 8, characterized in that it comprises means of illumination (48) of the product (1) in its support area (71) on the support structure (7). 10. A display device for commerce according to any of claims 7 to 9, characterized in that it comprises a support structure (7) in a substantially horizontal position, and broadcast means (40) of multimedia content (40a) in a substantially vertical position. 11. A display device for commerce according to any of claims 7 to 9, characterized in that it comprises a support structure (7) in a substantially horizontal position, and broadcasting means (40) of multimedia content (40a) in a substantially horizontal position. 12- Display device for commerce according to any of claims 1 to 6, characterized in that it comprises broadcasting means (40) of multimedia content (40a) associated with a product (1), in a substantially vertical position and at least one touch screen ( 40b) configured for interaction with the customer (2). 13- Display device for commerce according to any of claims 1 to 12, characterized in that it comprises means for collecting environmental data (49a) in the physical environment (3) of a display device, wherein the environmental data (49a ) are relative to at least one variable from the group of temperature, humidity, ambient noise and / or lighting. 14- Method of processing images captured (41a) by a display device for commerce, comprising the steps of: i) capture of captured images (41a) of the physical environment (3) of said store; ii) detection of a customer (2) of said business; iii) monitoring the physical position and / or movements and / or body movements and / or facial expression of said customer (2) of the store; iv) obtaining a plurality of metrics (91) associated with the physical position and / or the movements and / or the body movements and / or the facial expression of said customer (2) of the store; v) determination of the degree of interest (92) of the customer (2) in at least one product (1) presented in the display device based on said metrics (91). 15- Image processing method captured (41a) by a display device for commerce according to claim 14, characterized in that the monitoring of the body movements and / or the facial expression of the client (2) comprises the monitoring of at least one variable of the group of yaw, pitch, roll, proximity and time looking at a capture means (41) of captured images (41a). 16- Method of processing images captured (41a) by a display device for commerce according to claims 14 or 15, characterized in that the determination of the degree of interest (92) of the customer (2) in at least one product (1) presented in the display device it is based on a linear combination (93) of the number of stops with gaze in front of the display device, with the number of interactions with the product (1), and with the number of touches on a touch screen (40b) and / or with the number of interactions in a virtual reality representation device (6).