CN110266984A - A pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine - Google Patents

Abstract

The invention discloses a pan-tilt camera intelligent analysis teaching recording and broadcasting integrated machine, which includes a pan-tilt and a camera unit mounted on the pan-tilt; the camera unit includes: a camera for collecting audio and video signals of a teaching scene; a host processing module, It is used to receive audio and video signals, encode and encapsulate the audio and video signals. On the one hand, the encapsulated audio and video are stored locally, and on the other hand, the audio and video are released in the form of streaming media through the 4G network module; the video analysis DSP module is used for Receive the video signal, classify and recognize the target in the video signal, output the motion control signal to the pan/tilt control module according to the result of classification and recognition, and output the camera focus signal to the camera control module, so as to track the target in real time; the present invention uses the camera Integrated with the cloud platform, the identification and analysis of specific target behaviors in the classroom scene, combined with the real-time transmission of the 4G network, will help to further improve the intelligence of the teaching classroom.

Description

A pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine

technical field

The invention relates to the field of intelligent teaching, in particular to a pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine.

technical background

With the continuous development of computer vision and network streaming media technology, coupled with the continuous maturity of cloud computing, mobile Internet and big data analysis technology, its deep integration with modern education has given birth to a new generation of intelligent teaching recording and broadcasting system.

The intelligent recording and broadcasting system generally adopts the method of separating the camera from the recording and broadcasting host, that is, the front end uses an independent camera to obtain video data, and inputs the video into the recording and broadcasting host through SDI and HDMI, etc. After the recording and broadcasting host processes the video data, a On the one hand, the video is stored, and on the other hand, the network broadcast is performed through WiFi or Ethernet. Most of the live broadcasts in this mode are limited to LAN live broadcasts.

The camera is separated from the recording and broadcasting host, resulting in a large space occupation and insufficient compact structure; the network live broadcast mode based on WiFi or Ethernet has greatly restricted its applicable scenarios; in the past, the recording and broadcasting host was only responsible for recording and live broadcasting, and did not integrate The corresponding intelligent recognition algorithm will take additional time in post-processing.

Contents of the invention

In view of the above problems, the purpose of the present invention is to provide a cloud platform camera intelligent analysis teaching recording and broadcasting all-in-one machine, which adopts an integrated solution, which can realize real-time live broadcast and classroom on-demand in the wide area network, and at the same time can also Realize the automatic analysis of goals and behaviors in the classroom scene and complete the real-time tracking of the movement of teachers and students.

Concrete technical scheme of the present invention is as follows:

A pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine, comprising a pan-tilt and a camera unit mounted on the pan-tilt;

The camera unit includes a camera, a host processing module 4G network module, a video analysis DSP module, a camera control module, and a cloud platform control module;

The camera is used to collect video signals and audio signals of teaching scenes;

The host processing module is used to receive video signals and audio signals, encode the video and audio, and package them. On the one hand, the packaged audio and video are stored locally; ;

Described video analysis DSP module, is used for receiving video signal, carries out classification recognition to the target in video signal, according to the result of classification recognition, output motion control signal to cloud platform control module, output camera focusing signal to camera control module, thereby to Real-time tracking of targets;

The camera control module is used to control the focus of the camera;

The pan-tilt control module is used to control the motion of the pan-tilt.

Furthermore, in the video analysis DSP module, the classification and identification of targets includes the identification of teachers and students and the identification of specific behaviors of teachers and students in classroom scenes.

Further, for teachers, the specific behaviors include writing on the blackboard, asking questions, and wandering; for students, the specific behaviors include raising hands, standing up and sitting down.

Further, the real-time tracking of the target includes the tracking of the teacher's behavior or the tracking of the student's behavior; the tracking of the teacher's behavior mainly includes the tracking movement of the pan-tilt camera when the teacher is wandering; the tracking of the student's behavior mainly includes when the student is standing. A close-up of the camera's zoom and a release of the camera's zoom while the student is seated.

Further, the classification and recognition of the target in the video signal is realized by training a corresponding classifier, and the specific method is as follows:

S1: Train the classifier:

S11: Obtain the image training sample data set in the classroom scene. The data set includes various action information that needs to be recognized. Action information: teacher writing on blackboard, teacher asking questions, teacher wandering, students raising hands, students standing up, students sitting down;

S12: Extract sample features;

S13: Make a sample training set, and correspond the features extracted from the samples to the labels;

S14: Establish a classifier model;

S15: Perform classifier training on the sample training set, and the termination condition is to reach a predetermined accuracy or reach a predetermined number of training times;

S2: Use the trained classifier to classify and recognize classroom behavior actions:

S21: Obtain real-time video, and obtain a moving target area;

S22: Perform feature extraction on the moving target area;

S23: Use the classifier trained in step S1 to classify the extracted features to obtain classification results, and corresponding behavior categories can be obtained accordingly.

Further, in the S12, extracting sample features uses SIFT features, HOG features and LBP features.

Further, in S14, the classifier model adopts SVM classifier, K nearest neighbor classifier or Bayesian classifier.

Further, the camera unit also includes a local data transmission module, and the audio and video packaged by the host processing module are directly transmitted to the local display through the local data transmission module.

Further, the 4G network module can directly access the Internet, and users can directly access real-time streaming media through the external network.

Further, the above-mentioned pan-tilt is a suspended integrated pan-tilt, its horizontal movement adopts a worm gear transmission mode, and its pitch direction movement adopts a planetary gear transmission mode.

Compared with the prior art, the beneficial effects of the present invention are as follows:

1. The integrated recording and broadcasting host solution is adopted, which solves the large space occupation of the previous recording and broadcasting host system and the cumbersome installation and configuration process.

2. The 4G network is used for live streaming media transmission, which is more convenient for users to access anytime and anywhere.

3. Equipped with a dedicated teacher and student analysis camera in the classroom scene, intelligently tracking the behavior of teachers and students, which greatly improves the intelligence of the educational classroom.

Description of drawings

Fig. 1 is the overall structural diagram of the integrated platform;

Figure 2 is a schematic diagram of the disassembly of the integrated gimbal;

Fig. 3 is a schematic diagram of the integrated pan/tilt deceleration structure;

Fig. 4 is the specific structural block diagram of the mainboard of the integrated recording and broadcasting camera;

Fig. 5 is a structural schematic diagram of the mainboard of the integrated recording and broadcasting camera;

Figure 6 is a functional flow chart of the teaching recording and broadcasting all-in-one machine;

in:

1: Camera unit 2: PTZ

3: Pitch motion output end cover 4: Right end cover

5: Right side swing arm 6: PTZ tee

7: Connector 8: Bearing

9: Pan-tilt bracket 10: Worm gear shaft

11: Gimbal reducer 12: Left swing arm

13: Pitching motion power input 14: Horizontal motion power input

15: Pitching motion power output 16: Horizontal motion power output

Detailed ways

The present invention is described in more detail below in conjunction with accompanying drawing example:

As shown in FIG. 1 , a pan/tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine includes a pan/tilt 2 and a camera unit 1 mounted on the pan/tilt 2 .

According to the present invention of above-mentioned scheme, described cloud platform 2 is the integral cloud platform of suspended type, as shown in Figure 2, its horizontal motion adopts worm gear transmission mode, and worm gear is fixed on the worm gear shaft 10, and worm screw is relative to worm gear Rotating motion, that is, the horizontal rotation of the pan/tilt, wherein the worm gear shaft 10 on the pan/tilt bracket 9 and the bearing 8 adopt an interference fit, and the bearing 8 and the connecting piece 7 also adopt an interference fit, and the connecting piece 7 and the pan/tilt tee 6 Thread fit is adopted; the motion of the pan/tilt in the pitch direction adopts planetary gear transmission, and the planetary gear and the output end cover 3 of the pitching motion adopt an internal and external gear meshing mode. The rotation movement drives the left swing arm 12 and the right swing arm 5 to swing, thereby driving the camera unit 1 to do a pitching motion, and the right end cover 4 is used to assist in positioning the output end cover 3 of the pitching motion, and carries a certain load.

According to the present invention of above-mentioned scheme, described pan-tilt reduction box 11 is as the core component of integrated pan-tilt, and specific structure is as shown in Figure 3, and center position motor is used as pitch motion power input 13, drives planetary gear train motion, and planetary gear mechanism As the pitching motion power output 15, it drives the camera unit 1 to do the pitching motion; the eccentric position motor is used as the horizontal motion power input 14, and drives the worm to rotate as the horizontal motion power output 16, because the worm wheel is fixed, and finally the worm rotates around the worm wheel, that is, the camera unit 1 horizontal movement; both worm gear transmission and planetary gear transmission have the advantages of large transmission ratio and compact structure, which are very suitable for the application scene of the integrated pan/tilt.

According to the present invention of the above-mentioned scheme, the specific structural block diagram of the motherboard of the camera unit is as shown in Figure 4, the motherboard includes an audio signal interface, a video signal interface, a host processing module, a hard disk storage interface, a 4G network module, and a local data transmission module , video analysis DSP module, camera control module and pan-tilt control module; Described video signal and audio signal are obtained from camera respectively and delivered to host processing module, and host processing module encodes audio frequency and video, and the audio-video after encoding is by host The processing module continues to encapsulate it. On the one hand, the encapsulated video is stored locally through the hard disk storage interface, on the other hand, the network stream is pushed out through the 4G network module, and on the other hand, it can be directly transmitted through the local data transmission module without delay or loss. The video signal is transmitted to the local display; the video signal is transmitted to the video analysis DSP module at the same time, and the target in the video signal is classified and recognized. According to the result of classification and recognition, the motion control signal is output to the pan/tilt control module, and the camera focus signal is output to the camera control module. , so as to track the target in real time; the camera control module is used to control the focus of the camera; the pan-tilt control module is used to control the movement of the pan-tilt.

According to the present invention of above-mentioned scheme, provide a kind of structural schematic diagram of mainboard of integrated recording and broadcasting camera of preferred embodiment, as shown in Figure 5, be used for the host computer processing module of video processing function, network live broadcast function, recording storage function, video on demand function , the main processing chip is preferably Hisilicon HI3531A chip; the 4G network module used for 4G network function, the chip is preferably MT7620 series chip, and the data provided by the SIM card can achieve 4G network transmission; video analysis for video analysis function and motion tracking function DSP module, whose chip is preferably TMS320 series chip; the PTZ control module and camera control module used for PTZ control function, its chip is preferably STM32 series chip, which controls the related movement of PTZ and camera respectively through two RS232 channels; local data transmission module On the one hand, the data can be transmitted directly through the HDMI interface, on the other hand, the parallel output can be converted to serial output through the GV7600 chip, and then the data can be transmitted externally through the SDI interface; in addition, the main board of the camera provides a SATA interface and an SD card interface. For local video storage.

According to the present invention of above-mentioned scheme, described live network function is based on 4G network module access Internet, and the user can directly visit the real-time streaming media of this recording and broadcasting host through external network, and recording and broadcasting host adopts RTMP real-time streaming media protocol to transmit audio and video real-time streaming, The basic process is as follows:

S01: The recording and broadcasting host obtains real-time audio and video streams through the video processing module, and encodes them into a specific format, the audio is encoded into the AAC format, and the video is encoded into the H.264 format.

S02: Encapsulate the encoded audio and video data according to the FLV format, so as to conform to the RTMP transmission standard.

S03: Use the LibRTMP streaming media streaming framework to push the real-time stream to the RTMP streaming media server.

S04: RTMP streaming media player to play live streaming media.

According to the present invention of the above solution, the video analysis DSP module classifies and recognizes the targets in the video signal.

The classification and identification of targets includes the identification of teachers and students and the identification of specific behaviors of teachers and students in classroom scenarios.

For teachers, the specific behaviors include writing on the blackboard, asking questions, and wandering; for students, the specific behaviors include raising hands, standing up and sitting down.

For the behavior analysis of teachers and students in the classroom scene, it realizes the classification and recognition of behavior by training the corresponding classifier. The specific method is as follows:

S1: Train the classifier:

S11: Obtain a large number of picture training sample data sets in classroom scenes, which include various action information that needs to be recognized, such as teacher writing on the blackboard, teacher asking questions, teacher wandering, students raising their hands, students standing up, and students sitting down;

S12: Extract sample features, such as using SIFT features, HOG features, and LBP features.

S13: Make a training set of samples, and map the features extracted from the samples to the labels.

S14: Establish a classifier model, such as an SVM classifier, a K-nearest neighbor classifier or a Bayesian classifier.

S15: Perform classifier training on the sample training set, and the termination condition is reaching a predetermined accuracy or reaching a predetermined number of training times.

S2: Use the trained classifier for classroom behavior recognition:

S21: Obtain a real-time video, and acquire a moving target area by using a frame difference method or a background method.

S22: Perform feature extraction on the moving target area, such as SIFT features, HOG features, and LBP features, consistent with the sample production method.

S23: Use the classifier to classify the extracted features, obtain the classification result, and obtain the corresponding behavior category accordingly.

According to the present invention of above-mentioned scheme, described motion tracking function can be used for the tracking of teacher's behavior or the tracking of student's behavior; The tracking of described teacher's behavior mainly comprises the tracking movement of pan-tilt camera when teacher wanders; The tracking of described student's behavior , mainly including the zoom lens close-up of the camera when the student is standing and the zoom lens release of the camera when the student is sitting down. Motion tracking is realized based on behavior recognition. First, the corresponding behavior is obtained according to the video analysis DSP module, and the corresponding tracking type is executed according to the specific action type. Movement to achieve the effect of motion tracking; if the action type is for students to stand up, control the pan/tilt control module and camera control module at the same time. Zoom lens close-up, zoom in the lens to achieve the effect of close-up of students.

According to the present invention of the above-mentioned scheme, the flow chart of the implementation of each functional module of the teaching recording and broadcasting all-in-one machine is shown in Figure 6. When the all-in-one machine system is turned on, recording or on-demand can be selected; when the system selects recording, on the one hand The audio and video collected by the camera is encoded by the main processing unit, and then the audio and video are packaged into flv format video, which are used for local storage, webcast and local playback respectively. On the other hand, the video signal collected by the camera is used for video motion analysis, analysis The result is used for motion tracking; when the system selects on-demand, it accesses the local storage through the 4G network, selects the recording record to be watched, downloads the on-demand video to the local, and completes the playback.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the description of the present invention and the contents of the accompanying drawings, or directly or indirectly used in other related All technical fields are equally included in the scope of patent protection of the present invention.

Claims (10)

1. A cloud platform camera intelligent analysis teaching recording and broadcasting all-in-one machine is characterized in that it comprises a cloud platform and a camera unit mounted on the platform; The camera unit includes a camera, a host processing module 4G network module, a video analysis DSP module, a camera control module, and a cloud platform control module; The camera is used to collect video signals and audio signals of teaching scenes; The host processing module is used to receive video signals and audio signals, encode the video and audio, and encapsulate the audio and video. ; Described video analysis DSP module, is used for receiving video signal, carries out classification recognition to the target in video signal, according to the result of classification recognition, output motion control signal to cloud platform control module, output camera focusing signal to camera control module, thereby to Real-time tracking of targets; The camera control module is used to control the focus of the camera; The pan-tilt control module is used to control the motion of the pan-tilt. 2. a kind of pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine according to claim 1, is characterized in that, in the video analysis DSP module, the classification recognition to target comprises the identification of teacher and student and teacher and student Recognition of specific behaviors in classroom scenarios. 3. A kind of PTZ camera intelligent analysis teaching recording and broadcasting all-in-one machine according to claim 2, characterized in that, for teachers, its specific behaviors include teachers writing on the blackboard, teachers asking questions and teachers wandering; for students, its specific behaviors include student Raise your hands, stand up and sit down. 4. a kind of pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine according to claim 3, is characterized in that, carrying out real-time tracking to target comprises the tracking of teacher's behavior or the tracking of student's behavior; The tracking of described teacher's behavior mainly Including the tracking movement of the pan-tilt camera when the teacher is wandering; the tracking of the student's behavior mainly includes the close-up of the zoom lens of the camera when the student is standing and the release of the zoom lens of the camera when the student is sitting down. 5. according to claim 1-4 described in any one of claim 1-4, it is characterized in that, the object in the video signal is classified and identified, and the corresponding classifier is trained to To achieve, the specific method is as follows: S1: Train the classifier: S11: Obtain the image training sample data set in the classroom scene. The data set includes various action information that needs to be recognized. Action information: teacher writing on blackboard, teacher asking questions, teacher wandering, students raising hands, students standing up, students sitting down; S12: Extract sample features; S13: Make a sample training set, and correspond the features extracted from the samples to the labels; S14: Establish a classifier model; S15: Perform classifier training on the sample training set, and the termination condition is to reach a predetermined accuracy or reach a predetermined number of training times; S2: Use the trained classifier to classify and recognize classroom behavior actions: S21: Obtain real-time video, and obtain a moving target area; S22: Perform feature extraction on the moving target area; S23: Use the classifier trained in step S1 to classify the extracted features to obtain classification results, and corresponding behavior categories can be obtained accordingly. 6. A kind of pan-tilt-camera intelligent analysis, teaching, recording and broadcasting all-in-one machine according to claim 6, characterized in that, in said S12, extracting sample features uses SIFT features, HOG features and LBP features. 7. A kind of pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine according to claim 6, characterized in that, in said S14, the classifier model adopts SVM classifier, K-nearest neighbor classifier or Bayesian classifier. 8. A kind of pan-tilt camera intelligent analysis teaching recording and broadcasting all-in-one machine according to claim 1, characterized in that, the camera unit also includes a local data transmission module, and the audio and video after the host processing module encapsulation passes through the local data transmission module Transfer directly to local display. 9. The all-in-one machine for intelligent analysis, teaching, recording and broadcasting of PTZ cameras according to claim 1, wherein the 4G network module can directly access the Internet, and the user directly accesses real-time streaming media through the external network. 10. The all-in-one machine for intelligent analysis, teaching, recording and broadcasting of a pan-tilt camera according to claim 1, wherein the pan-tilt is a suspended integrated pan-tilt, and its horizontal motion adopts a worm gear transmission mode, The movement in the pitch direction adopts planetary gear transmission.