CN115953214A - Product category labeling method and system across online shopping platforms - Google Patents

Abstract

The invention discloses a method and system for labeling commodity categories across online shopping platforms, and belongs to the technical field of big data processing. The technical problem to be solved is how to simply and effectively realize cross-platform screening and label daily necessities. The method comprises the following steps: collecting commodity information of commodities in each online shopping platform, said commodity information at least including commodity names; for each category, collecting characteristic words of said category and constructing a set of characteristic words of said category; for each Category, collecting noise words of the category and constructing a set of noise words of the category; for each commodity collected from each online shopping platform, under each category, if and only if the product name of the commodity contains Any element in the feature word set of the category and any element in the noise word set not including the category is marked with a category label corresponding to the category for the commodity.

Description

Product category labeling method and system across online shopping platforms

technical field

The invention relates to the technical field of big data processing, in particular to a method and system for labeling commodity categories across online shopping platforms.

Background technique

Grain, oil, vegetables and other daily necessities are the focus of work to ensure supply and stabilize prices in the market. More and more people choose to buy daily necessities such as grain, oil and vegetables online. The daily necessities of various online shopping platforms are also key monitoring objects.

However, in general, there are many categories of online shopping platforms, and the platform categories of different online shopping platforms are quite different. Some platforms have products in the order of millions or tens of millions; some platforms have more than a thousand categories related to a certain target daily necessities; some platforms have three levels of categories, and some platforms have categories There are five levels; some platform categories are designed by sellers themselves. Even if both platforms have a three-level category structure, the platform categories corresponding to the same type of daily necessities are not the same.

How to simply and effectively implement cross-platform screening and labeling of daily necessities is a technical problem that needs to be solved.

Contents of the invention

The technical task of the present invention is to address the above deficiencies and provide a cross-online shopping platform commodity category labeling method and system to solve the technical problem of how to simply and effectively realize cross-platform screening and labeling of daily necessities.

In the first aspect, the present invention provides a method for labeling commodity categories across online shopping platforms. For commodities as daily necessities, it is used to mark commodity categories across online platforms based on Chinese keywords. The method includes the following steps:

Collect commodity information of commodities in each online shopping platform, where the commodity information includes at least commodity names;

For each category, collect the feature words of the category and build the set of feature words of the category;

For each category, collecting noise words of the category and constructing a set of noise words of the category;

For each product collected from each online shopping platform, under each category, if and only if the product name of the product contains any element in the feature word set of the category, and does not contain any element of the category Any element in the noise word set marks the product with a category label corresponding to the category.

Preferably, for each category, the characteristic words of the category are collected based on the name of the category itself, synonyms of the category and names of subcategories of the category.

As preferably, for each category, collecting the noise words of the category and constructing the noise word set of the category includes the following steps:

For each commodity that contains any element in the feature word set, randomly select a commodity of a predetermined type as a sample commodity;

For the sample product, find a product that does not belong to the category as the target product;

For the target commodity, search for noise words from the commodity names of the target commodity, and construct a noise word set of the category based on noise words of all target commodities.

Preferably, for each product collected from each online shopping platform, under each category, use regular expressions and REGEXP_SUBSTR() to determine the inclusion relationship, and judge whether the product meets the following conditions: if and only if the The product name of the product contains any element in the feature word set of the category, and does not contain any element in the noise word set of the category.

Preferably, the method also includes the steps of:

For each category, regularly collect the feature words of the category and update the set of feature words of the category;

For each category, regularly collect noise words of the category and update the set of noise words of the category;

For the marked commodities, under each category, the labeling of the commodities is updated based on the updated feature word set and noise word set of the category.

In the second aspect, the present invention is a cross-line shopping platform product category labeling system, which is used to perform product labeling through the cross-line shopping platform product category labeling method described in any one of the first aspects, and the system includes :

A commodity collection module, the commodity collection module is used to collect commodity information of commodities in each online shopping platform, and the commodity information includes at least commodity names;

A feature word set building module, for each category, the feature word set building module is used to collect the feature words of the category and build the feature word set of the category;

A noise word set building module, for each category, the noise word set building module is used to collect the noise words of the category and construct the noise word set of the category;

Commodity labeling module, for each product collected from each online shopping platform, the product labeling module is used to perform the following: under each category, if and only if the product name of the product contains the characteristics of the category Any element in the word set and any element in the noise word set that does not contain the category at the same time, mark the category label corresponding to the category for the commodity.

Preferably, for each category, the characteristic word set building module is used to collect the characteristic words of the category based on the name of the category itself, synonyms of the category and names of subcategories of the category.

As preferably, for each category, the noise word set building module is used to perform the following implementation to collect the noise words of the category and construct the noise word set of the category:

For each commodity that contains any element in the feature word set, randomly select a commodity of a predetermined type as a sample commodity;

For the sample product, find a product that does not belong to the category as the target product;

For the target commodity, search for noise words from the commodity names of the target commodity, and construct a noise word set of the category based on noise words of all target commodities.

Preferably, for each product collected from each online shopping platform, under each category, the product labeling module is used to judge the inclusion relationship through regular expressions and REGEXP_SUBSTR(), and judge whether the product satisfies the following Condition: if and only if the product name contains any element in the feature word set of the category, and does not contain any element in the noise word set of the category.

Preferably, the system further includes an update module, and the more detailed module is used to execute:

For each category, regularly collect the feature words of the category and update the set of feature words of the category;

For each category, regularly collect noise words of the category and update the set of noise words of the category;

For the marked commodities, under each category, the labeling of the commodities is updated based on the updated feature word set and noise word set of the category.

The product category labeling method and system across online shopping platforms of the present invention have the following advantages: for each category, a feature word set and a noise word set are constructed, and for each product, only the product name information of the product needs to be obtained, and in each Under the category, based on the combination of feature words and the set of noise words, it is possible to screen whether the product belongs to the category and mark it. When there are large differences in the platform categories between different online shopping platforms, the product labeling is realized.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present invention. For some embodiments, those skilled in the art can also obtain other drawings based on these drawings without creative effort.

The present invention will be further described below in conjunction with the accompanying drawings.

Fig. 1 is a flowchart of a method for labeling commodity categories across online shopping platforms in Embodiment 1.

Detailed ways

The present invention will be further described below in conjunction with accompanying drawing and specific embodiment, so that those skilled in the art can better understand the present invention and can be implemented, but the embodiment given is not as the limitation of the present invention, in the case of no conflict Next, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

Embodiments of the present invention provide a method and system for labeling commodity categories across online shopping platforms, which are used to solve the technical problem of how to simply and effectively realize cross-platform screening and labeling of daily necessities.

Example 1:

The present invention is a method for labeling commodity categories across online shopping platforms. For commodities as daily necessities, it is used to mark commodity categories across online platforms based on Chinese keywords. The method includes the following steps:

S100. Collect commodity information of commodities in each online shopping platform, where the commodity information includes at least commodity names;

S200. For each category, collect characteristic words of the category and construct a set of characteristic words of the category;

S300. For each category, collect noise words of the category and construct a noise word set of the category;

S400. For each product collected from each online shopping platform, under each category, if and only if the product name of the product contains any element in the feature word set of the category and does not contain the Any element in the noise word set of the category is labeled with a category label corresponding to the category for the commodity.

In this embodiment, in step S200, a set of characteristic words of a category is determined. As a specific implementation example, for each category, the characteristic words of the category are collected based on the name of the category itself, synonyms of the category, and names of subcategories of the category. For example, for "grain-rice", the set of category feature words may be {rice, japonica rice, indica rice, northeast rice, Wuchang rice, long-grain rice}.

Step S300 determines the set of noise words of each category, as a specific implementation example, for each category, collects the noise words of the category and constructs the set of noise words of the category, including the following steps:

(1) For each commodity that contains any element in the feature word set, randomly select a commodity of a predetermined type as a sample commodity;

(2) For the sample product, find a product that does not belong to the category as the target product;

(3) For the target commodity, search for noise words from the commodity names of the target commodity, and construct a noise word set of the category based on noise words of all target commodities.

That is, in this step, randomly select a small number of commodities (for example, 10% or 100 of the total amount) from commodities that contain any element of the category feature word set; check the commodities that do not actually belong to the category, from their A set of noise words is summarized in the product title. For "grain-rice", the set of category noise words can be {rice noodles, rice noodles, rice vinegar, rice cakes, rice crisps, sesame}.

Step S400 marks the commodities. As a specific implementation example, for each commodity collected from each online shopping platform, under each category, using SQL statements as an example, the inclusion relationship is judged through regular expressions and REGEXP_SUBSTR() , to determine whether the product satisfies the following condition: if and only if the product name of the product contains any element in the feature word set of the category and does not contain any element in the noise word set of the category. There are similar functions in other mainstream languages such as Python.

As an improvement of this embodiment, the method also includes the following steps:

For each category, regularly collect the feature words of the category and update the set of feature words of the category;

For each category, regularly collect noise words of the category and update the set of noise words of the category;

For the marked commodities, under each category, the labeling of the commodities is updated based on the updated feature word set and noise word set of the category.

In the above improved step S500, the set of characteristic words and the set of noise words of each category are dynamically adjusted. According to the actual situation, regularly adjust the feature word set and noise word set of each category to screen and label new products or update products that have been marked.

There are many categories of online shopping platforms, and the platform categories of different online shopping platforms are quite different. This application method is simple and universal. You only need to obtain the name information of the product, and you can select and mark the target necessities of life from many products on each platform, paving the way for the next step of the work of ensuring supply and stabilizing prices for relevant departments.

Example 2:

The present invention is a product category labeling system across online shopping platforms, including a product collection module, a feature word set building module, a noise word set building module, and a product labeling module. The system is used to label products through the method disclosed in Embodiment 1 .

The commodity collection module is used to collect commodity information of commodities in each online shopping platform, and the commodity information includes at least commodity names.

For each category, the characteristic word set building module is used to collect the characteristic words of the category and construct the characteristic word set of the category.

As a specific implementation example, for each category, the feature word set building module is used to collect feature words of the category based on the name of the category itself, synonyms of the category, and names of subcategories of the category. For "grain-rice", the set of category feature words can be {rice, japonica rice, indica rice, northeast rice, Wuchang rice, long-grain rice}.

For each category, the noise word set building module is used to collect noise words of the category and construct a noise word set of the category.

As a specific implementation example, the noise word set building module is used to perform the following implementation to collect the noise words of the category and construct the noise word set of the category:

(1) For each commodity that contains any element in the feature word set, randomly select a commodity of a predetermined type as a sample commodity;

(2) For the sample product, find a product that does not belong to the category as the target product;

(3) For the target commodity, search for noise words from the commodity names of the target commodity, and construct a noise word set of the category based on noise words of all target commodities.

In this example, for commodities containing any element of the category feature word set, the noise word set building block is used to randomly select a small number of commodities (for example, 10% or 100 of the total amount); Items, summarizing the set of noise words from their item names. For "grain-rice", the set of category noise words can be {rice noodles, rice noodles, rice vinegar, rice cakes, rice crisps, sesame}.

For each product collected from each online shopping platform, the product labeling module is used to perform the following: under each category, if and only if the product name of the product contains any element in the feature word set of the category , while not including any element in the noise word set of the category, and labeling the product with a category label corresponding to the category.

As a specific implementation example, for each product collected from various online shopping platforms, under each category, the product labeling module is used to judge the inclusion relationship through regular expressions and REGEXP_SUBSTR() to determine whether the product The following conditions are met: if and only if the commodity name of the commodity contains any element in the feature word set of the category, and does not contain any element in the noise word set of the category. Specifically, in the SQL language, the judgment of the "contains" relationship can be implemented using regular expressions and "REGEXP_SUBSTR()", and other mainstream languages such as Python have similar functions.

As an improvement, the system of this embodiment also includes an update module, and the more detailed module is used to execute:

(1) For each category, regularly collect the feature words of the category and update the set of feature words of the category;

(2) For each category, regularly collect the noise words of the category and update the set of noise words of the category;

(3) For the marked commodities, under each category, the labeling of the commodities is updated based on the updated feature word set and noise word set of the category.

In this embodiment, the update module regularly adjusts the set of feature words and the set of noise words of each category according to the actual situation, and is used to screen and label new products or update products that have been marked.

The present invention has been shown and described in detail through the accompanying drawings and preferred embodiments above, but the present invention is not limited to these disclosed embodiments, and those skilled in the art based on the above-mentioned multiple embodiments can know that the above-mentioned different embodiments can be combined More embodiments of the present invention can be obtained by means of code review in the present invention, and these embodiments are also within the protection scope of the present invention.

Claims (10)

1. A commodity class labeling method of an online shopping platform is characterized in that commodities which are taken as living necessities are labeled based on Chinese keywords, and the method comprises the following steps:

collecting commodity information of commodities in each online shopping platform, wherein the commodity information at least comprises commodity names;

for each category, collecting the characteristic words of the category and constructing a characteristic word set of the category;

for each category, collecting interference words of the category and constructing an interference word set of the category;

for each commodity collected from each online shopping platform, under each category, if and only if the commodity name of the commodity contains any element in the feature word set of the category and does not contain any element in the interference word set of the category, labeling the commodity with a category label corresponding to the category.

2. The method of claim 1, wherein the characteristic words of each item are collected based on its own name, its synonym, and its subclass name.

3. The method for labeling commodity categories on the cross-line shopping platform according to claim 1, wherein for each category, the method for collecting the interference words of the category and constructing the interference word set of the category comprises the following steps:

randomly selecting a commodity of a preset type as a sample commodity for each commodity containing any element in the feature word set;

for the sample commodity, searching for a commodity which does not belong to the category as a target commodity;

and for the target commodity, searching interference words from the commodity name of the target commodity, and constructing an interference word set of the category based on the interference words of all the target commodities.

4. The method for labeling merchandise categories of the online shopping platform according to claim 1, wherein for each merchandise collected from each online shopping platform, under each category, the inclusion relation judgment is performed through a regular expression and REGEXP _ SUBSTR (), and whether the merchandise satisfies the following conditions is judged: and if and only if the commodity name of the commodity contains any element in the characteristic word set of the category and does not contain any element in the interference word set of the category.

5. The method for labeling merchandise items on an off-line shopping platform according to any one of claims 1 to 4, further comprising the steps of:

for each category, periodically collecting the characteristic words of the category and updating the characteristic word set of the category;

for each category, periodically collecting interference words of the category and updating an interference word set of the category;

and for the marked commodities, updating the marks of the commodities based on the updated feature word set and the updated interference word set of the commodity under each commodity.

6. A system for labeling commodities based on an online shopping platform, characterized in that the system is used for labeling commodities by the method for labeling commodities based on the online shopping platform according to any one of claims 1 to 5, and the system comprises:

the system comprises a commodity acquisition module, a commodity display module and a commodity display module, wherein the commodity acquisition module is used for acquiring commodity information of commodities in each online shopping platform, and the commodity information at least comprises commodity names;

the characteristic word set building module is used for collecting the characteristic words of the categories and building a characteristic word set of the categories for each category;

the interference word set building module is used for collecting interference words of the categories and building an interference word set of the categories for each category;

a merchandise labeling module for executing the following for each merchandise collected from each online shopping platform: and under each category, if and only if the commodity name of the commodity contains any element in the feature word set of the category and does not contain any element in the interference word set of the category, labeling the commodity with a category label corresponding to the category.

7. The system of claim 6, wherein for each item, the feature word set construction module is configured to collect feature words of the item based on an item name, an item synonym, and an item subclass name.

8. The system of claim 6, wherein for each category, the interfering word set constructing module is configured to perform the following operations to collect interfering words of the category and construct the interfering word set of the category:

randomly selecting a commodity of a preset type as a sample commodity for each commodity containing any element in the feature word set;

for the sample commodity, searching for a commodity which does not belong to the category as a target commodity;

and for the target commodity, searching interference words from the commodity name of the target commodity, and constructing an interference word set of the category based on the interference words of all the target commodities.

9. The system for labeling merchandise items on an online shopping platform according to claim 6, wherein for each merchandise collected from each online shopping platform, in each item, the merchandise labeling module is configured to perform containment relationship judgment through a regular expression and REGEXP _ SUBSTR () to determine whether the merchandise satisfies the following condition: and if and only if the commodity name of the commodity contains any element in the feature word set of the category and does not contain any element in the interference word set of the category.

10. The system for labeling merchandise items on an off-line shopping platform according to any one of claims 6 to 9, further comprising an updating module for performing:

for each category, periodically collecting the characteristic words of the category and updating the characteristic word set of the category;

for each category, periodically collecting interference words of the category and updating an interference word set of the category;

and for the marked commodities, updating the marks of the commodities based on the updated feature word set and the updated interference word set of the commodity under each commodity.

Images