WO2006005947A1

WO2006005947A1 - Testing device and method

Info

Publication number: WO2006005947A1
Application number: PCT/GB2005/002739
Authority: WO
Inventors: Anthony Richardson; David Edwards; Malek Ladki
Original assignee: Sensustech Limited
Priority date: 2004-07-12
Filing date: 2005-07-11
Publication date: 2006-01-19
Also published as: WO2006005947A8

Abstract

There is provided a method of obtaining information about a communications network, the method comprising: placing at least one network testing device in a coverage area of the network; downloading call criteria to the or each network testing device; determining, in the or each network testing device, based on the downloaded criteria, when calls should be made over the network; initiating the calls over the network; measuring a network quality parameter during the calls; measuring a position of the network testing device at a time of each of the network quality parameter measurements; and transferring the measured network quality parameters and the measured positions to a central data store.

Description

TESTING DEVICE AND METHOD

Technical Field of the Invention

This invention relates to a method of testing one or more communications networks, and more specifically to a method of testing one or more networks using testing devices.

Background of the Invention

It is useful for a communications network operator to obtain accurate and detailed information about the performance of the communications network. For example, this information might relate to the network users' ability to make calls without delay, and the available quality of the users' connections over the network.

In order to obtain such information, it is known to place test equipment within the coverage area of a communications network, and then to place calls to and from the test equipment. Often, these calls are placed at specific time intervals. This can produce a limited amount of data which is useful for the network operator. However, as the quality of the network can vary according to the time of day and particular geographical location, the data acquired by the prior art test equipment is often not a true representation of the situation faced by real network traffic.

Summary of the Invention

Therefore, according to a first aspect of the present invention, there is provided a method of obtaining information about a communications network, the method comprising: placing at least one network testing device in a coverage area of said network; downloading call criteria to the or each network testing device; determining, in the or each network testing device, based on said downloaded criteria, when calls should be made over said network; initiating said calls over said network; measuring a network quality parameter during said calls; measuring a position of said network testing device at a time of each of said network quality parameter measurements; transferring said measured network quality parameters and said measured positions to a central data store; and setting subsequent call criteria based on the measured network quality parameters transferred to said central data store. Therefore, according to a second aspect of the present invention, there is provided a network testing device, adapted to be moved in a coverage area of a communications network, the device being adapted to: receive downloaded call criteria; determine, based on said downloaded criteria, when calls should be made over said network; initiate said calls over said network; measure a network quality parameter during said calls; measure a position of said network testing device at a time of each of said network quality parameter measurements; and transfer said measured network quality parameters and said measured positions to a central data store.

According to a further aspect of the present invention, there is provided a method of obtaining information about a mobile communications network, the method comprising: placing at least one network testing device in a coverage area of said network; initiating a call to or from said network testing device in said network; transmitting a first voice data sequence over said network to said network testing device in a first time period; measuring a first network quality parameter from the first voice data sequence received at said network testing device; transmitting a second voice data sequence over said network from said network testing device in a second time period contemporaneous with the first time period; measuring a second network quality parameter from the second voice data sequence received over said network; measuring a position of said network testing device at a time of each of said network quality parameter measurements; and transferring said measured first network quality parameter and said measured position to a central data store.

Brief Description of the Drawings

For a better understanding of the present invention, and to show more clearly how it may be carried into effect, reference will now be made, by way of example, to the following drawings, in which:

Figure 1 shows a communications system and a plurality of network testing devices in accordance with the invention;

Figure 2 shows a network testing device in accordance with the invention; Figure 3 shows a server in accordance with the invention; and Figure 4 is a flow chart illustrating the method of obtaining information about a mobile communications network in accordance with the present invention.

Detailed Description of the Preferred Embodiments

Figure 1 shows a communications system and a plurality of network testing devices. In accordance with the invention, the network testing devices 2 are placed in a coverage area of a communication network 4, so that calls may be made to and from the devices 2 over the network 4 and network quality measurements can be taken during the calls. In this illustrated embodiment of the invention, the network testing devices 2 are operated by the operator of the mobile communication network 4, in order to provide information about the performance of the network 4.

Although the invention will be described further below with reference to a mobile communication network, it will be appreciated that the invention is applicable to many other types of communication network.

As used herein, "calls" includes voice calls, data calls, text messages sent using the Short Message Service (SMS), messages sent using the Multimedia Messaging Service (MMS) or any other type of communications traffic provided by the network 4.

The network 4 may be any type of mobile communications network, for example a GSM, EDGE, GPRS or 3G network.

Preferably, the network testing devices 2 are placed on objects which move in the coverage area of the network 4. For example, a network testing device 2 could be given to a delivery worker to carry on his round. This means that the network testing device 2 would move through the coverage area of the network 4 on a substantially predetermined route, allowing test calls to be placed in different locations in the network 4.

Alternatively, the network testing devices 2 may be placed on objects that move on routes that have been predetermined by the network operator.

Alternatively, the network testing devices 2 may be placed on objects, such as taxis, that move on substantially random routes throughout the coverage area of the network 4. In this case, the network operator has little or no control over where the network testing device 2 will move next.

Alternatively, the network testing device 2 may be placed in a fixed location within the coverage area of the network 4. For example, the network testing device 2 may be placed in a railway station. Preferably, each of the network testing devices 2 has a position measurement system for determining its position within the network 4. Preferably, the Global Positioning System (GPS) is used to determine the positions of each of the devices 2.

The mobile communication network 4 is connected to the internet 6, which is in turn connected to one or more servers 8 and one or more clients 10. The one or more servers 8 and one or more clients 10 communicate with each other via the internet 6.

Figure 1 also shows a network testing device 2a, which is located at a fixed position within the coverage area of the network 4. The network testing device has a fixed connection to the internet 6. Any testing device may have a fixed or wireless connection to the internet, independent of the connection over the mobile communication network 4, in order to allow it to transmit data.

Although Figure 1 shows a client 10 having a fixed line connection to the server 8 over the internet, it will be apparent that the client 10 may have a wireless connection to the internet, or may be connected to the server 8 by a local area network or other connection.

The servers 8 download call criteria to the network testing devices 2, and the devices 2 make test calls and collect network quality measurements and other information relating to the sequence of events, including, but not limited to, network signalling information, on the basis of the downloaded criteria, as will be described in more detail below.

The test calls may include device-to-device calls, calls to the server 8 from the testing device 2, or calls from the server 8 to the testing device 2. The server 8 may make and receive calls directly over the mobile communication network 4, or may make and receive calls over a public switched telephone network (PSTN) 11. The server 8 uses a fixed call generator (FCG) 12 for generating and receiving test calls to or from the testing devices 2 over the public switched telephone network 11. The FCG 12 may form part of the server 8 or may be a physically separate unit. The server 8 and the FCG 12 may be connected via voice and signalling links over a local or wireless area network. Once the network testing devices 2 have collected a number of network quality measurements, the measurements and related data (such as the position of the device 2 at the time each of the measurements was taken) are uploaded from the network testing devices 2 to the one or more servers 8 via the internet 6.

The measurements and related data can be uploaded to the server immediately, or when it is possible to transfer data to the server. The network testing devices 2, 2a therefore contain memories for storing the data while awaiting the possibility to transfer data to the server.

In addition to storing the uploaded measurements, the servers 8 then analyse and output the results of the measurements to the one or more clients 10. Figure 2 shows in more detail a network testing device 2 in accordance with the invention. The network testing device comprises an antenna 13 connected to radio transceiver circuitry 14 for transmitting signals to, and receiving signals from, the mobile communication network 4. A network authentication device 16, such as a Subscriber Identity Module (SIM) card, is connected to the transceiver circuitry 14 for allowing the device 2 to operate in the network 4.

A controller 18 is connected to the transceiver circuitry 14 for controlling the operation of the network testing device 2. The device 2 further comprises a voice codec 20 for encoding voice samples for transmission by the transceiver circuitry 14, and a memory 22. The controller 18 in the device 2 is responsible for taking the measurements of the network quality during the test calls. These measurements may include the received signal strength, the signal-to-noise ratio of a received signal, the strength of signals in neighbouring cells of the network 4, information relating to the cell serving the device 2, the rate of data throughput, the bit error rate in received data, the quality of the voice sample received or the number of calls dropped. Where the received test call is a voice call, the received voice signal may be recorded for subsequent playback. The measurements may also include detecting whether specific services are available to the device at that time. It will be appreciated that there are many other measurements that may be taken by the controller 18 to analyse the performance of the network 4.

The measurements of the network quality parameters by the controller 18 may be made once during each call, at discrete points during the call, or for a period of time during the call.

The controller 18 is also responsible for measuring the location of the device 2 when the device 2 includes a position measurement system.

The measurements taken by the controller 18 are stored in the memory 22. Stored measurements are then transferred to a server 8, either periodically or at the conclusion of each test call.

Figure 3 shows a server 8 in accordance with the invention. The server 8 comprises a terminal manager module 24 for receiving and managing connections from the network testing devices 2 for the upload of the network quality measurements via the internet 6, downloading call pattern criteria to the devices 2 and synchronising the time in the network.

The server 8 further comprises a traffic generation manager module 26 for generating the call pattern criteria. The criteria may be determined by a network operator, or may be determined on the basis of previous records of network traffic.

The call criteria are set dynamically. That is, with an initial set of criteria having been determined, the criteria for a particular network testing device are then modified dynamically during use of the system, based on the position of the network testing device relative to the positions of the other network testing devices, and based on the received network quality measurements. For example, if the number of network testing devices in a small area changes, then the number of calls that each device needs to make will also change, in order to obtain the desired number of measurements.

Further, if the received network quality measurements indicate the appearance of a problem in the network, call criteria can be set for the network testing devices in order to allow additional testing of this problem.

A database 28 in the server 8 stores the configuration information for the test system, including data relating to the identity of the network testing devices 2, system users and network configuration, and also stores the measurements uploaded from the network testing devices 2. The server 8 also comprises a client manager module 30 for managing sessions established between the server 8 and one or more clients 10 and providing the clients with usable results derived from the uploaded measurements.

Figure 4 shows a flow chart illustrating the method of obtaining information about a mobile communications network in accordance with the present invention. In step 101 , the call pattern criteria determined in the server 8 is downloaded to the network testing device 2.

In step 103, the network testing device 2 determines when a call should be made or received over the network 4 on the basis of the downloaded criteria and measurements of the time, location of the device 2 and/or recent events in the network 4.

If the measurements satisfy the downloaded criteria, the network testing device 2 initiates a call over the network 4 or prepares to receive and incoming call (step 105).

The type of call, destination/origin, duration and content will be determined from the call pattern criteria received from the server 8. In step 107, the network testing device 2 measures one or more network quality parameters during the call. In step 109, the network testing device 2 measures its position in the network 4 at the time each network quality parameter measurement is taken.

In step 111 , in accordance with a further determination by the device 2 using the criteria, the measured network quality parameters and measured positions are transferred to a central data store.

In step 113, the server sets the criteria for subsequent calls. As mentioned above, the call criteria are set dynamically, and thus previously set criteria can be modified, based for example on measurements received from the set of network testing devices. Thus, in accordance with the invention, the network testing devices 2 test the network 4 by placing and receiving calls to or from other network testing devices 2 or to or from the server 8. As indicated above, the server 8 may make or receive calls to or from the devices 2 directly over the mobile communications network 4, or via a public switched telephone network (PSTN) 11 using the fixed call generator 13. As the levels and types of traffic in the network 4 may vary considerably throughout the day, and may vary between different geographical regions in the network 4, the collection of quality data for the network 4 is controlled so that the pattern of calls established to or from the network testing devices 2 is representative of real traffic in the network 4. In addition, the pattern of calls can be set in order to test the performance of the network during some predicted future event. For example, if it is known that a large crowd of people will attend an event in the coverage area of the network, it can be predicted that there will be heavy network traffic in the relevant part of that area at a time in the future. The pattern of calls from the devices can then be set in order to simulate the number and type of calls to be expected from such a crowd. The performance of the network in response to the test calls can then be used in order to plan for the event.

As mentioned above, the one or more servers 8 manage the network testing devices 2. In accordance with the invention, the servers 8 generate a call pattern strategy for each of the network testing devices 2 in the network 4. This call pattern strategy is a set of criteria that is evaluated regularly by each network testing device 2 to determine whether to place or receive a test call over the network 4.

In accordance with the invention, the criteria in the call pattern strategy provided by the server 8 may require the device 2 to place or receive test calls over the network 4 based on the level of call traffic generally present in the network 4 at that time. For example, the network 4 may experience a relatively high level of call traffic at a certain time of day. Therefore, a network testing device 2 in the coverage area of the network 4 at that time may make more test calls, or calls more often, over the network 4 relative to the number of test calls or the rate at which test calls are placed when there is a normal or lower level of call traffic. This will provide the network operator with additional data for the quality of the network for that period of time.

Further in accordance with the invention, the criteria in the call pattern strategy provided by the server 8 may require the device to place or receive test calls over the network 4 based on the location of the network testing device 2, as particular regions of the network 4 may experience higher levels of call traffic than others.

For example, more calls are likely to be placed in parts of the network 4 that cover urban areas than parts of the network 4 that cover isolated rural areas.

Therefore, when a network testing device 2 determines that it is in an area of the network 2 that experiences higher levels of call traffic relative to other areas of the network 4, the rate at which test calls are placed by the device 2 can be increased.

Conversely, if the network testing device 2 is in an area of the network 4 that experiences low levels of call traffic, the rate at which test calls are placed by the device 2 can be reduced.

Further in accordance with the invention, the criteria in the call pattern strategy provided by the server 8 may require test calls to be placed or received based on the occurrence of certain events within the network 4.

For example, if a user is making a call and the call is dropped or fails to connect, the user will often try to reconnect the call straight away. Therefore, the device 2 may be operated to place another test call over the network 4 if the previous test call was dropped before completion or if it failed to connect.

Instead of, or as well as, specifying how to modify the rate at which test calls are made or received over the network 4, the criteria in the call pattern strategy provided by the server 8 may indicate which other network testing device 2 in the network 4 that a call should be placed to or received from, the type of test call to make (e.g. voice, data, text message or picture message), the number of test frames in a test voice call, the gap between frames within a test voice call, or the minimum length of time between making two successive test calls.

It will be appreciated that the above criteria can be combined in many ways to determine a call pattern strategy for each network testing device 2. For example, the call pattern strategy may specify an increased frequency of test calls in a particular area of the network 4 at a particular time, or may require more test text messages than data calls to be sent over the network 4 in a particular area.

To receive test calls, the test devices 2 can operate in one of three modes. In the first mode, the server 8 can download an 'expected call pattern' to each of the test devices 2 that indicates when the devices 2 can expect to receive incoming test calls from other devices 2 or the server 8. If no test call is received, the device 2 can report a failed call to the server 8.

Secondly, the test devices 2 can wait for any incoming calls. This means that the test device 2 will wait in an idle mode until a test call is received. Thirdly, the test device 2 makes outgoing test calls according to the downloaded call pattern criteria, and can receive test calls during any intervals between the test calls.

In the preferred embodiment of the invention, the servers 8 contain data relating to typical traffic patterns in the network 4. The call pattern strategies for the devices 2 can then be devised such that the test calls provide a statistically valid test of the network performance under real traffic conditions.

Preferably, the call pattern strategy provided to the network testing devices 2 by the servers 8 includes further criteria for allowing the devices 2 to determine when to upload the measurements of the network quality to the servers 8. Preferably, the network testing devices 2 regularly upload their position measurements to the server 8. Using the position measurements of a testing device 2, the server 8 reviews the call pattern strategy determined for that device 2. If the server

8 determines that a different call pattern strategy is required for that device 2, the new call pattern strategy is downloaded to the device 2. The device 2 then uses the new call pattern criteria to determine when to make and receive test calls over the network

4.

Preferably, the network testing device 2 does not require any input from a user to place or receive test calls, and automatically determines when to place and receive calls over the network 4 in accordance with the criteria provided by the server 8 and on measurements of the time, position of the network testing device 2 and/or recent network events.

When the device 2 is placing a test voice call, the voice codec 20 retrieves one or more of a predefined set of voice samples from the memory 22 and encodes them for transmission by the transceiver circuitry 14. The voice codec 20 also decodes voice samples received in incoming test calls and passes them to the controller 18 for analysis. The present invention can also be used to test the network quality experienced by a particular corporate customer of the network operator. In this embodiment, the call pattern strategy provided to the network testing devices 2 will be devised such that the test calls established by the testing devices 2 will correspond to the pattern of calls made by the corporate customer. For example, it is likely that employees of a corporate customer (who each have their own mobile terminal) will make more calls in urban areas where their offices are located. It is therefore important for the network operator to ensure that the corporate customer is experiencing an appropriate quality of service. In this case, the call pattern strategy can be tailored to cause more test calls to be generated in the areas in which the corporate customer has offices.

The present invention can also be used to test the quality of new network services before they are released for use by customers of the network operator. In addition, the effect of the introduction of the new service on the quality of existing network services can be determined. The measurement of the quality of a new network service can be performed by specifying call criteria as described above, and by establishing test calls to and from appropriately equipped network testing devices 2.

In order to measure the effect of the introduction of the new service on the quality of existing network services, separate test calls can be initiated using the new service and the existing service at substantially the same time. Measurements of the quality of the existing service can be taken as described above. The server 8 can then compare the quality measurements with measurements taken previously to see if the introduction of the new service has had an adverse effect on the quality of the existing service.

In a mobile communication network, the characteristics of the transmit and receive paths of the communication channel can often be very different to each other. Therefore, in a preferred embodiment of the present invention, when a network testing device 2 establishes a voice test call over the network 4 with another network testing device 2, both devices 2 transmit test voice samples to the other device 2. This allows the network operator to concurrently test the transmit and receive paths of the communication channel established with the network 4 and characterise the communications link over a particular time period.

Similarly, where a network testing device 2 establishes a test voice call with a device on a fixed line, the transmit and receive paths of the communication channel can be tested by both the network testing device 2 and the device on the fixed line transmitting voice samples to the other device. In a further embodiment of the present invention, the controller 18 may request that one or more network settings are modified, including the strategy for allocation of radio network resources, to measure the effect of their variation on the performance of the network 4. For example, these settings may include the power with which a signal is transmitted from a base station antenna or the frequency used for the transmit or receive channel. It will be appreciated that there are many network settings that may be varied depending on the type of network and the particular equipment used to implement the network. Specifically, in accordance with the preferred embodiment, the device 2 may measure the performance of the network 4 and, on the basis of these measurements, determine that one or more network settings should be modified. The device 2 may then request that the network 4, or a part of the network 4, be reconfigured according to the new settings. Once the new settings have been implemented in the network 4, the device 2 may perform further measurements on the performance of the network 4. After these measurements have been taken, the network 4 may continue operating using the new settings, or may be reset to the original settings.

In an alternative embodiment of the invention, the servers 8 may analyse the uploaded measurements obtained by the devices 2, and determine whether to modify the network settings. If it is determined that the settings should be modified, the servers 8 can then modify the settings and request the devices 2 in the network 4 to perform further measurements.

In a further embodiment of the invention, a network testing device 2 is capable of testing more than one communications network. The device 2 may test a first network according to the above methodology, and then test a second network using the same method. In order for the network testing device 2 to test two or more communication networks, the device is provided with a network authentication device 16 for each of the networks it is required to test. If the additional networks are different types and use different transmission protocols, the network testing device 2 will be provided with appropriate transceiver circuitry 14 for transmitting signals to, and receiving signals from, those mobile communication networks.

There is therefore provided a testing device and a method of testing one or more communications networks.

Claims

1. A method of obtaining information about a communications network, the method comprising: placing at least one network testing device in a coverage area of said network; downloading call criteria to the or each network testing device; determining, in the or each network testing device, based on said downloaded criteria, when calls should be made over said network; initiating said calls over said network; measuring a network quality parameter during said calls; measuring a position of said network testing device at a time of each of said network quality parameter measurements; transferring said measured network quality parameters and said measured positions to a central data store; and setting subsequent call criteria based on the measured network quality parameters transferred to said central data store.

2. A method as claimed in claim 1 , comprising placing a plurality of said network testing devices in said coverage area of said network.

3. A method as claimed in claim 1 or 2, comprising placing said at least one network testing device on objects which move in said coverage area of said network.

4. A method as claimed in claim 3, comprising placing said plurality of said network testing devices on objects which move on substantially predetermined routes in said coverage area of said network.

5. A method as claimed in claim 4 comprising placing said plurality of said network testing devices on objects which move on substantially predetermined routes over a significant proportion of said coverage area of said network.

6. A method as claimed in claim 3, comprising placing said plurality of said network testing devices on objects which move on substantially random routes in said coverage area of said network.

7. A method as claimed in claim 1 or 2, comprising placing said plurality of said network testing devices at fixed locations within said coverage area of said network.

8. A method as claimed in any preceding claim, wherein the step of downloading call criteria to the or each network testing device comprises downloading location data to the or each network testing device, such that the or each network testing device determines that calls should be made over said network when the network testing device is at a particular location.

9. A method as claimed in any preceding claim, wherein the step of downloading call criteria to the or each network testing device comprises downloading time data to the or each network testing network testing device, such that the or each network testing device determines that calls should be made over said network at particular times.

10. A method as claimed in any preceding claim, wherein the step of downloading call criteria to the or each network testing device comprises downloading data to the or each network testing device relating to events that occur in the network, such that the or each network testing device determines that calls should be made over said network when said events occur.

11. A method as claimed in any preceding claim, wherein, when a voice call is initiated between the network testing device and another device over the network, the method further comprises: transmitting a first voice data sequence over said network to said network testing device in a first time period; measuring a first network quality parameter from the first voice data sequence received at said network testing device; transmitting a second voice data sequence over said network from said network testing device in a second time period contemporaneous with the first time period; measuring a second network quality parameter from the second voice data sequence received over said network; and measuring a position of said network testing device at a time of each of said network quality parameter measurements.

12. A method as claimed in any preceding claim, the method further comprising: determining the call criteria on the basis of typical traffic patterns in the network.

13. A method as claimed in any of claims 1 to 11 , the method further comprising: determining the call criteria on the basis of typical traffic patterns for a user or group of users in the network.

14. A method as claimed in claim 2, comprising setting said subsequent call criteria based on the measured network quality parameters transferred to said central data store from a plurality of said network testing devices.

15. A method as claimed in claim 2, comprising setting said subsequent call criteria based on the measured positions of a plurality of said network testing devices.

16. A method as claimed in any preceding claim, the method further comprising: analysing said measured network quality parameters and said measured positions to determine modifications to one or more settings of the network; modifying the one or more settings of the network in accordance with said analysis; initiating further calls over said network by the or each network testing device in accordance with the downloaded call criteria; measuring a network quality parameter during said further calls.

17. A method as claimed in claim 16, wherein said modification of one or more settings of the network comprises modifying the allocation of radio network resources.

18. A network testing device, adapted to be moved in a coverage area of a mobile communications network, the device being adapted to: receive downloaded call criteria; determine, based on said downloaded criteria, when calls should be made over said network; initiate said calls over said network; measure a network quality parameter during said calls; measure a position of said network testing device at a time of each of said network quality parameter measurements; and transfer said measured network quality parameters and said measured positions to a central data store.

19. A network testing device as claimed in claim 18, further comprising: a memory, for storing said measured network quality parameters and said measured positions before transfer to the central data store.

20. A network testing device as claimed in claim 18 or 19, wherein said device is adapted to transfer said measured network quality parameters and said measured positions to the central data store over said mobile communications network.

21. A network testing device as claimed in claim 18 or 19, wherein said device is adapted to transfer said measured network quality parameters and said measured positions to the central data store over an internet connection independent of said mobile communications network.

22. A system for obtaining information about a communications network, the system comprising: a server and at least one network testing device, wherein the server is adapted to: determine call criteria for at least one network testing device; download the call criteria to the or each network testing device; and store uploaded network quality parameter measurements from the or each network testing device in a central data store; and the or each network testing network testing device is adapted to: determine, based on said downloaded criteria, when calls should be made over said network; initiate said calls over said network; measure a network quality parameter during said calls; measure a position of said network testing device at a time of each of said network quality parameter measurements; and transfer said measured network quality parameters and said measured positions to said server, wherein said server is adapted to determine said call criteria based on the measured network quality parameters transferred from the network testing device.

23. A method of obtaining information about a mobile communications network, the method comprising: placing at least one network testing device in a coverage area of said network; initiating a call to or from said network testing device in said network; transmitting a first voice data sequence over said network to said network testing device in a first time period; measuring a first network quality parameter from the first voice data sequence received at said network testing device; transmitting a second voice data sequence over said network from said network testing device in a second time period contemporaneous with the first time period; measuring a second network quality parameter from the second voice data sequence received over said network; measuring a position of said network testing device at a time of each of said network quality parameter measurements; and transferring said measured first network quality parameter and said measured position to a central data store.