US20030110249A1 - System and method for monitoring key performance indicators in a business - Google Patents
System and method for monitoring key performance indicators in a business Download PDFInfo
- Publication number
- US20030110249A1 US20030110249A1 US09/877,414 US87741401A US2003110249A1 US 20030110249 A1 US20030110249 A1 US 20030110249A1 US 87741401 A US87741401 A US 87741401A US 2003110249 A1 US2003110249 A1 US 2003110249A1
- Authority
- US
- United States
- Prior art keywords
- data
- modified
- format
- alert
- communicatively coupled
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
Definitions
- the present invention pertains to systems and methods for measuring, quantifying, and monitoring business performance and operations, and more particularly, to systems and methods for analyzing key business metrics in a business environment and for providing a visual snapshot of the business metrics specified by a user.
- a system for monitoring business performance indicators in a networked environment comprises a data source having a predefined format, an agent communicatively coupled to the data source, wherein the agent is configured according to the data source format and wherein the agent is operative to gather data from the data source and translate the data into a first modified format thereby creating modified data.
- the system also comprises a reaper communicatively coupled to the agent and configured to retrieve the modified data from the agent, a data repository communicatively coupled to the reaper and configured to store the modified data an alert detector communicatively coupled to the data repository and configured to compare the modified data with a first configuration parameter, and a dashboard controller communicatively coupled to the data repository and configured to display the modified data in a format defined by a second configuration parameter.
- a system for monitoring business performance indicators in a networked environment comprises a data source having a predefined format, an agent communicatively coupled to the data source, wherein the agent is configured according to the data source format and wherein the agent is operative to gather data from the data source and translate the data into a first modified format thereby creating modified data.
- the system also comprises a reaper communicatively coupled to the agent and configured to retrieve the modified data from the agent, a repository manager communicatively coupled to the reaper, a data repository communicatively coupled to the repository manger, an alert detector communicatively coupled to the repository manager, and a dashboard controller communicatively coupled to the repository manager.
- a method for monitoring a business metric in a networked environment comprises coupling to a data source having a known format, wherein the data source includes data that represents the business metric, configuring an agent according to the data source format, gathering the data from the data source via the agent, translating the data into a first modified format, storing the modified data in a data repository, comparing the modified data with an alert parameter range, displaying the modified data in a format defined by a second configuration parameter, determining whether the modified data falls within the alert parameter range, and producing an alert if the modified data falls within the alert parameter range.
- a system for monitoring a business metric in a networked environment comprises a processor, a data storage device, and an instruction set residing on the data storage device, wherein the instruction set is configured to perform a method comprising coupling to a data source having a known format, wherein the data source includes data that represents the business metric, configuring an agent according to the data source format, gathering the data from the data source via the agent, translating the data into a first modified format, storing the modified data in a data repository, comparing the modified data with an alert parameter range, displaying the modified data in a format defined by a second configuration parameter, determining whether the modified data falls within the alert parameter range, and producing an alert if the modified data falls within the alert parameter range.
- a system for monitoring business performance indicators in a networked environment comprises a collector adapted to communicatively coupled to a data source having a predetermined format, wherein the collector is configured according to the data source format and wherein the collector is operative to gather data from the data source and translate the data into a first modified format thereby creating modified data, a data manager communicatively coupled to the collector and configured to manage the input and output of the modified data between the collector and a data storage device, wherein the data manager is adapted to communicatively couple with an alert device, and a display interface communicatively coupled to the data manager and configured to display the modified data in a format defined by a second configuration parameter.
- FIG. 1 is an architecture diagram of a dashboard system constructed in accordance with an embodiment of the present invention
- FIG. 2 is an architecture diagram of a dashboard system constructed in accordance with another embodiment of the present invention.
- FIG. 3 is a screenshot of a dashboard in accordance with an embodiment of the present invention.
- FIGS. 4 - 10 are various details of the dashboard of FIG. 3;
- FIGS. 11 and 12 are sample dashboards for an organization utilizing an association-franchise relationship.
- FIG. 1 shows a general architectural diagram of a dashboard system 100 constructed in accordance with an embodiment of the present invention.
- the dashboard system 100 is one module of an overall business monitoring and reporting system that is utilized to present a user with an overview of business metrics and other key performance indicators.
- the dashboard will sometimes be referred to herein as the VisionPortTM Dashboard System. VisionPortTM is a trademark of XOR, Inc.
- the dashboard system 100 provides a high level executive view of key performance indicators for an eBusiness.
- the dashboard system 100 preferably presents a visually oriented snapshot of the status of key business metrics identified by a user.
- the dashboard system 100 is an open and extensible framework of computer software or fixed hardware components designed to enable organizations to build real-time measurement and monitoring solutions for applications and systems.
- the dashboard can be installed at a user's own facility or maintained at an off-site Application Services Provider for a particular client. Additionally, the various components of the dashboard system 100 maybe spread across different platforms or physical locations.
- the dashboard system 100 is preferably configurable on a per-user or per-client basis, customizable to show each user or client exactly what information they want access to. For instance, a marketing group can be shown one set of informnation such as sales and advertising revenues while a company officer can be shown a broader scope of infonnation about the performance of the company.
- the dashboard system 100 is adapted to either link to or direct a user to an appropriate 3 rd party application tool to get the required information.
- the dashboard system 100 is preferably viewable through readily available internet browser software, such as current versions of Internet Explorer® or Netscape Navigator®.
- dashboard system 100 Other devices such as WAP-compatible phones, Palm OS based devices, Blackberry, VoiceXML, and email programs are also preferably compatible with the information provided by the dashboard system 100 so that a user can access the information from a variety of interface devices.
- the framework of the dashboard system 100 is preferably extensible in order to support a wide range of application interfaces.
- the diagram depicted in FIG. 1 provides an overview of the technical architecture of a preferred dashboard system framework and its system requirements.
- the dashboard system 100 includes four primary components, a reaper 120 , one or more agents 140 , an alert detector 160 , and a dashboard controller 180 . While each of these four primary components will be described in more detail below, the following is an overview of their function and structure.
- the agents 140 are used to gather information from various data sources, third party applications, data servers, legacy systems, etc., and reformat the data into a common scheme using XML.
- the reaper 120 is adapted to contact the agents in order to gather the reformatted data and store the reformatted data in a data repository (reference number 124 in FIG. 1).
- the reaper 120 is also preferably adapted to perform data aggregation and history creation.
- a repository manager 190 is also shown in FIG. 1 that further manages the inflow and outflow of data to and from the data repository 124 .
- the alert detector 160 reads the data from the repository and compares the information to an alert configuration parameter. If an alert is detected, the alert detector initiates an appropriate action such as an SNMP trigger, paging, email, or visual alert.
- the dashboard controller 180 acts as the interface between an end-user and the information in the data repository. The dashboard controller requests information from the data repository 124 and converts the requested information into a format compatible with the user's selected interface. The dashboard controller 180 is also adapted to read a configuration file in order to determine which information should be delivered to the user interface.
- Data is retrieved via the agents 140 from a number of different data sources 141 .
- the data sources 141 can be presented in any number of different formats, including, without limitation, a client-specific data source 142 , a pre-supported data source 144 , a datamart 146 , a web log analysis tool 148 , and a third party application 150 .
- a plurality of agents 140 are utilized, one for each of the data sources being incorporated into the dashboard system 100 .
- Each of the agents 140 are specifically formatted to interact with its associated data source.
- the agents are computer software programs written specifically for each of the respective data sources.
- the reaper 120 communicates with each of the agents 140 by a two way communication link 152 and is adapted to pull the data gathered by the agents on a scheduled basis.
- the reaper is adapted to utilize, for example, an XML push/pull routine in order to access and retrieve the data from the reaper 120 .
- An agent polling configuration file 122 defines the necessary polling schedule information for the reaper 120 .
- a datastore or data repository 124 serves as a data cache and stores the information gathered by the reaper 120 .
- a repository manager 190 includes one or more memory caches 192 , and further monitors and directs data that is transferred between the dashboard controller 180 , the data repository 124 , the reaper 120 , and the alert detector 160 .
- An alert detector 160 is in two-way communication with the repository manager 190 .
- the alert detector traps threshold conditions that are defined in an alert configuration file 162 and compares the threshold conditions to the data stored in the data repository 124 and processed by the repository manager 190 .
- the alert detector 160 also sends notifications or alarms to a user based on definitions set in the alert configuration file 162 .
- Notifications and alarms can be in the form of an SNMP trap, email notification, pager notification, visual notification, or any other type of data transfer alert. For instance, if a data value stored in the data repository 124 is outside of a range defined in the alert configuration file, the alert detector will send an alarm or notification to the user indicating the “out of range” condition.
- the dashboard system 100 is monitoring web-site hits per hour and the user wants to know if this value rises above a number that corresponds to the maximum capacity of the web server, the alert detector will recognize this condition and send an alert to the user.
- the dashboard controller 180 is communicatively coupled to the repository manager 190 and generally functions to format the visualization of the data gathered by the agents 140 and passed through the reaper 120 .
- a unique per-user configuration file 182 is customized for each user of the dashboard system 100 and defines for each user what information from the dashboard system 100 should be displayed to the user on a display device.
- the display device can be either a conventional PC monitor, a cell phone display, a PDA display (such as a device operating under the PALM or Windows CE operation systems), a pager display, or any other type of data display device. Additionally, information may be translated into a VoiceXML so that information may be accessed on a voice based system.
- the dashboard controller 180 includes an HTML interface 188 that visually formats the data for display on a conventional web browser in the form of an HTML dashboard 184 .
- the dashboard controller 180 also includes another interface 191 that formats the data for display on another type of dashboard such as one adapted to display or otherwise communicate WML, VoiceXML, or another data protocol. In this sense, it is contemplated that the dashboard controller 180 may include any number of interfaces.
- the dashboard controller 180 is preferably adapted to be expandable to incorporate interfaces for future types of dashboard viewing tools.
- Each of the components of the dashboard system architecture depicted in FIG. 1 are preferably implemented in computer software residing on some form of fixed or otherwise permanent storage system such as a data server or other network of computers. However, the dashboard system 100 may also be implemented in a fixed hardware format such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA).
- ASIC Application Specific Integrated Circuit
- FPGA Field Programmable Gate Array
- FIG. 2 is another embodiment of a dashboard system 101 constructed in accordance with the present invention.
- the dashboard system 101 is similar in structure and function to the dashboard system 100 previously described except that a repository manager is not utilized. Instead, the reaper 120 is directly coupled to the data repository 124 which is in turn coupled to the dashboard controller 180 .
- the dashboard system 100 collects data through its respective agents 140 .
- an agent is a program that fetches a set of information, formats the information into XML, or some other form of open coding language for storage in the data repository 124 .
- Agents can be created using programming libraries developed from software languages such as PERL and JAVA as well as other programming languages.
- Each agent 140 can return one or more data items can be designated to return information in one of two ways. First, each of the agents 140 can return information dynamically. In the the reaper 120 contacts the agent program via a Hyper Text Transfer Protocol (HTTP). The agent 140 then dynamically gathers the information and delivers XML to the reaper 120 . Second, the agent can return information statically.
- HTTP Hyper Text Transfer Protocol
- Agents can be formatted to get their associated data through a variety of means including SQL queries, text file processing, HTML scraping via an HTTP call, incoming email processing, SNMP, or spreadsheet exports. Other data gathering means are contemplated by the present invention and the above list is not meant to be limiting.
- Appendix A which is hereby incorporated by reference, contains a listing of exemplary agents that have been developed for use in accordance with a dashboard system constructed in accordance with an embodiment of the present invention. Appendix A also includes selected information from future agents that are contemplated by a dashboard system constructed in accordance with an embodiment of the present invention.
- an agent When an agent generates its data, it preferably converts that data into XML or another open code environment.
- the XML format is used for communication between the reaper 120 and the agents 140 , and from the repository manager 190 to the dashboard controller 180 .
- dashboard Data Types Generally, each piece of data assembled by the dashboard must conform to a data type. “Is a” notation is preferably used to declare inheritance between the different data types. When one data type “is a”, it inherits the required and optional attributes from the given data type.
- Appendix B which is hereby incorporated by reference, contains examples of data types supported by a dashboard system constructed in accordance with an embodiment of the present invention.
- Appendices A and B only represents examples of agents and dashboard data types created for specific applications. It is contemplated that agents and data types may be created for any type of existing data or third party data processing application. Customized libraries can be established for use in creating agents. For example, PERL libraries may be established to supply methods for creating new dashboard objects, “freezing objects into XML”, “thawing” objects into objects, and “selecting” a data item from within an XML structure. An example PERL library is included at Appendix C which is hereby incorporated by reference.
- agents can be adapted to have the ability to “push” data to the reaper, rather than the data being polled. This may be accomplished by having the reaper supply an agent with a “callback” URL. The agent can then hit the callback URL to supply data to the reaper.
- An optional sequence/version number may be added to the dashboard XML item. Agents that write static HTML files would update the version number each time they write the file. Then, the reaper can use the version number to determine if the dataset has changed. Agents can also be adapted to have the ability to accept arguments from the reaper. For example, a generic “SQL Agent” could be created that accepts a SQL statement from the reaper and returns the results.
- the reaper gathers XML data by contacting the agents.
- the agents are contacted based on information supplied through a reaper configuration file (agent polling configuration file 122 from FIG. 1).
- the reaper configuration file supplies agent information for all agents, across all clients or there may exist multiple reaper configuration files; one for each client.
- Reaper Configuration File Description Indicates an individual agent that should be contacted to gather information. Notes The name of each agent in the top level, or in a group (see below) must be unique. If an object disappears from the XML tree returned by an agent, it should be removed from the repository.
- Attributes Name The name of the XML file that the XML tree returned by this agent will be stored in.
- source_url The URL of the agent that supplies the XML data. Enabled Indicates whether this agent is currently being refreshed (used). Valid values are true and false.
- Default: true Path The name of the directory that the XML file will be stored in. If not specified, will inherit from its group (in which case it must be set in the group; see below).
- Refresh The refresh frequency in seconds or a cron-style entry (see man 5 crontab). If not specified, will inherit from its group (in which case it must be set in the group; see below).
- rate_type Indicates that the reaper should automatically generate a rate field for this agent.
- the rate is calculated after new data is stored and the history is updated. If not specified, will inherit from its group (if set in the group). Valid values are none, difference: number, percent: number, persecond: number, median: number, averagechange: number, and averagevalue: number. Default: none. Precision Specifies a precision to limit all number elements to. A precision of “1” will make numbers look like “2.5”. Default: leaves numbers as-is. History Indicates that the reaper should accumulate history for items within the XML tree returned by the agent. The history value specifies how many historical values should be kept. Default: 0
- rate_types that may be requested in the above configuration file example are further defined as follows: none Do not populate the rate field.
- difference: number Rate is calculated as: (current_value ⁇ history[number]). For example, difference: 1 calculates the difference between the current and previous values. If number is greater than the size of the history, then the oldest item in the history will be used instead. percentage: number The rate_type is calculated in the same manner as difference, but a percentage change is placed in the rate field instead of the difference.
- median: number Rate is calculated as: (current_value + history[number])/2. For example, median: 5 calculates the median between the current and 6th history value.
- persecond number Rate is calculated as: (current_value ⁇ history[number])/ (current_timestamp ⁇ timestamp[number]). For example, persecond: 1 calculates the difference between the current and previous in units per second. average: number Calculates the average of the history items, from current to history[number]. Averagechange: number Calculates the average change, from current to history[number].
- An element will not be pushed onto the history if its current timestamp is the same as the most recent item in the history list. This would occur when the agent program is setting the timestamp explicitly. For example, an agent may choose to set the timestamp to the time that the agent knows the data was last updated (with web logs, that is likely the timestamp of the previous day). If an agent provides its own history, then that history should be used instead, and not be overwritten.
- An element of history length of X actually stores X+1 elements in its history.
- the “zero” element in the history array is the same as the current value/timestamp.
- Elements 1 to X+1 are the historical values/timestamps.
- Agents may be classified and categorized by groups. Each group preferably has the following characteristics associated with it. Description Specifies a group of related agents. Optionally specifies that the reaper should aggregate multiple items together into a single item. Notes The name of each group must be unique. Attributes name A unique name that can be used to refer to this group. Also used as the XML file name where the aggregated numbers will be stored if aggregation is enabled. enabled Indicates whether this group is currently being refreshed (used). Valid values are true and false. If false, then none of the agents in the group will be refreshed, independent of the settings of their individual flags. If true, then each agent can specify its own enabled status.
- Default: true path The directory location in which to store the resulting aggregated numbers, if aggregation is enabled (see below). If a path is defined here, it will be used as a default for agents in the group that don't have a path defined. refresh The refresh frequency in seconds or a cron-style entry (see man 5 crontab). If specified, it will be used as a default for agents in the group that don't have a refresh defined. precision Specifies a default precision to limit all number elements to in the group. A precision of “1” will make numbers look like “2.5”. Default: leaves numbers as-is. rate_type Indicates that the reaper should automatically generate a rate field for the agents in this group.
- Valid values are none, merge, and overlay. Default: none Aggregate Indicates the allowable time offset in seconds to use when Maxoffset aggregating history items. Default: infinite Aggregate Indicates that a rate of change should be calculated for the Rate_type aggregated values. See the “rates” section below for possible values. Default: none
- old_xml.overlay(new_xml) would overlay the contents of old_xml with the contents of new_xml. Any elements that are in new_xml that are not in old_xml will also be kept. Anything in the intersection of the two is defaulted to new_xml.
- the actual number of items summed will be used to calculate the average. In other words, if there are 5 agents in the group, but one agent does not return a particular value, then the remaining 4 values will be summed and divided by 4 to calculate the average. If the agent wishes a “zero” value to be included in the average, it should return an item with an explicit value of 0.
- An error will occur if items to be aggregated do not have exactly the same type (e.g. it will be an error to try to aggregate text and image or number and gauge).
- the individual XML trees retrieved from each agent will be stored as in the repository, so that a UI can retrieve the individual values as well as the aggregate.
- the historical values will be merged into a single list.
- the reaper will do some level of error checking, as defined by aggregate_maxtimeoffset, to make sure that the timestamps are not different by more than this value. If they are, the history item with the newer timestamp is used, and a warning is logged.
- the reaper may supply the agent with a “callback” URL. The agent can then hit the callback URL to supply data to the reaper.
- the reaper may be adapted to recognize the version number to determine if the dataset has changed.
- the reaper may be adapted to support attribute(s) to send arguments to an agent that is designed to be “generic.” Exceptions may be indicated if the reaper is not able to contact an “enabled” agent. For example, an email may be sent to an administrator.
- the reaper may also be adapted so that the reaper children that are performing a request can “time out.” This timeout value would default to something reasonable (i.e. 60 seconds) and may be overwritten by an agent/group specific “timeout” attribute.
- the controller can be adapted to make a dynamic request for information. If the reaper does not have the request in the datastore, the reaper would then contact the appropriate agent for the information.
- a configuration file for each client may be set up and a configuration file for “generic” information (e.g.: News, Stock Market Quotes) may also be set up.
- the reaper should be able to be told to re-read a specified configuration file.
- An “alert” can be set on any Dashboard indicator and is triggered when an indicator does, for example, one of the following: 1) descends below a certain threshold; 2) rises above a certain threshold; or 3) is on or between two numbers.
- an alert is defined, a severity and an action are associated with it. For example, severity levels of zero through three can be defined (this can be expanded upwards if needed). Actions that can be taken include: 1) Visual: An element in the indicator changes, such as an arrow turning red; 2) Email: An email is sent to a specified address.
- This address could be a normal user's address, a list of users or a pager email address (most alpha-numeric pagers have associated email addresses); 3) SNMP: An SNMP “trap” can be triggered.
- SNMP stands for “Simple Network Management Protocol” and is used by network monitoring software to gather information and alerts about routers, switches, machines and other network-enabled devices. For example, SNMP could be used to alert a user's monitoring software. In order for the controller to know if an alert has been triggered, the Alert Detector writes out an XML file that contains the appropriate alert information.
- an alert configuration file supplies alert information for all agents, across all clients or there may exist an alert configuration file for each user or client.
- the following describes the contents of the configuration file that are wrapped with an alerts XML tag.
- Action Description Indicates an action to take when an alarm is triggered. Notes The name of each action must be unique. Attributes name The name of the action. This is used in the monitor tags to indicate which action to take when an alert is triggered.
- Type The type of action to take. Can be one of: display: Indicate that an alert has happened on the dashboard display email: Email a notification to someone that an alert has occurred snmp: Trigger an SNMP trap destination The contents of the destination depends on the action type. display: The alert file to write its alert XML to.
- Email Email address(es) of the people to notify. Note that this can also be a pager email address snmp: MIB color Only set for actions of type “display”. The color is made available to the templating engine.
- Level Description Defines the actual alert ceiling/floors for this monitor.
- a level may only be specified inside a monitor. Notes The name of each level within its monitor must be unique. Attributes name The name of the action. This is used in the monitor tags to indicate which action to take when an alert is triggered.
- Eff_type Effective start date specific as YYYYMMDD. If one is not specified, it will begin immediately. Low If both a low and high are set, the low specifies the floor for this alert. If an alert falls on or between the low and high (inclusive), this alert is triggered. If only a low is set, then an alert is triggered if the value falls on or below the low. High If both a low and high are set, the high specifies the ceiling for this alert.
- Result Description Defines tags to set in the template engine for this level.
- a result can only be specified inside a level.
- Attributes directional Tag used in the templating engine to indicate part of the name of a directional graphic to display.
- Msg Message to display to the end-user when they place their mouse over the alert icon. Severity The severity level for this alert. Ranges can be 0 through 3. 0 is typically is visual-only alert (change a directional graphic red, for example), while alerts 1 through 3 typically indicate a more severe problem. An alert icon is displayed to the user for severity levels 1 through 3.
- Log Description Defines the action that this alert uses. Attributes action Indicates the action name to trigger for this alert.
- destination Destination XML file overwrites the destination specified in the action.
- the above XML alert is set up to behave in the following manner. If the change in the index since the market opened is positive, no alert is set. If the change in the index since the market opened is between 0 and ⁇ 25, alert severity 0 is triggered, and the down-arrow is made red. If the change in the index since the market opened is between ⁇ 25 and ⁇ 50, alert severity 1 is set, the down-arrow is made red and an “Alert Severity 1 ” icon is displayed on the indicator. If the user places their mouse over the indicator they will see the text “Down over 25”. Alert severity 2 is set when the change in the index since the market opened is between ⁇ 50 and ⁇ 75.
- the down-arrow is made red and an “Alert Severity 2 ” icon is displayed on the indicator. If the user places their mouse over the indicator they will see the text “Down over 50”. If the change in the index since the market opened is over ⁇ 75, alert severity 3 is set, the down-arrow is made red, and an “Alert Severity 3 ” icon is displayed on the indicator. If the user places their mouse over the indicator they will see the text “Down over 75”.
- the alert detector may have the following additional features. Users can have the ability to specify Key Performance Indicators.
- the controller gets its information from a user/client-specific output file and there are user/client-specific alarm configuration files. A single alarm file may “include” another. Duplicate alarms would be overwritten by the alarms in the included file. Alarms may activate/deactivate based on a start and end date and time (i.e.: today the ceiling is 10, tomorrow the ceiling is 20). Alarms may be dynamically set by requesting ceiling/floor information from another data source.
- a user interface may be created for changing alarm information and alerts may be set that span different agents.
- the dashboard controller configuration is composed of a large set of files that can be configured per user. These include HTML/ASCII templates, data location definitions, and the display configuration.
- the basic configuration is composed of the following directory hierarchy: dashboard
- the conf directory contains the XML files that define the screen display.
- Each user has one configuration file preferably named username.xml.
- the templates directory contains a default directory, and may contain a directory hierarchy for each user.
- the default directory is used if a template file can't be found in the user directories.
- These directories contain templates, ending in .tmpl, specific for the display renderers.
- Each type of renderer contains a set of templates that correspond to either the type of dashboard object or a template type specified in the user configuration.
- the templates are processed to replace a set of tags with the data values from the data sources. There are five tags that are used to do this:
- TMPL_VAR and TMPL_IF tags can be contained within the loop.
- the loop is ran until it has iterated over all of the data values contained in the name (as specified in the user configuration file).
- Data configuration files contain XML entities that define aliases (shortcuts) to the Dashboard data repository items. These aliases are used in the controller configuration files.
- the default data file is preferably located in a file labeled /info/dashboard/ conf/full/data.xml, but there may be one file per client/user (named the client/user).
- the aliases add an abstraction layer that may be able to be used to reduce the overall number of templates that are needed.
- Load Description Associates an alias name to a data file Notes
- a load should be created for each XML data file that will be accessed in the user's display configuration.
- Attributes Name The name of the alias.
- the name of each load should be unique.
- File The full filename of the repository data file. In a future release, this should be changed to be relative to the base directory of the dashboard repository.
- select A path into the specified file's XML data tree. This attribute allows an alias to point to a data node that is deep within the XML tree. If a select is not specified, then the alias simply points to the data file.
- Each display configuration file defines the display of a specific user or client. They work with the aliases defined in the data configuration.
- the configuration file contains the following hierarchy of elements which are described in more detail in Appendix D, which is hereby incorporated by reference: - navorg - section - construct - item
- All of the tags can contain one or more attr elements, which behave similar to an entity/attribute that is included within the main element tag. This is primarily used for readability, when an item has a lot of attributes.
- a dashboard controller utilized in a dashboard system constructed in accordance with an embodiment of the present invention may also be adapted to include the following features.
- Object caching may be added in order to speed up the display.
- “Include” files may be added inside the Controller configuration files, so that similar per-user configuration files can be easily made. Included files would overwrite duplicate elements. Display thresholds for KPIs (Key Performance Indicators) and alerts may be graphically displayed and alternate graphing engines may be used.
- the following description provides a preferred embodiment of a dashboard system constructed in accordance with the present invention.
- the example is representative of an implemented Dashboard system as designed and built by XOR, Inc. of Boulder, Colo.
- the dashboard is preferably password protected and can be customized for different executives, departments or partners of an eBusiness.
- WAP Wireless Application Protocol
- FIG. 3 represents an exemplary main dashboard screen 300 of a dashboard system constructed in accordance with an embodiment of the present invention.
- the dashboard screen 300 is broken down into a grid consisting of up to sixteen indicators although more or less indicators may be utilized. As shown in FIG.
- the sixteen indicators are labeled (and described) as 305 (control panel), 310 (Dow Jones Stock Market Indicator), 315 (NASDAQ stock market ticker), 320 (S&P 500 stock market ticker), 325 (web site statistics), 330 (non-web site statistics), 335 (advertising statistics), 340 (revenue statistics), 345 (previous day web site statistics-yesterday), 350 (previous day web site statistics-week ago), 355 (average web site statistics), 360 (1 month average web site statistics), 365 (home page load time statistics), 370 (average system load time statistics), 375 (web page visits per day), and 380 (registered users).
- a blank box is displayed whenever there are fewer than 16 indicators in use (e.g.
- Appendix G which is hereby incorporated by reference, provides a table that describes of each of the above indicators in further detail.
- the dashboard Upon opening in a browser-based environment, the dashboard preferably will resize the browser window to the dimensions necessary to display the indicators.
- the uppermost left indicator 305 is referred to as a control panel and is preferably present on all dashboard embodiments.
- FIG. 4 shows the control panel 305 in further detail.
- the control panel 305 contains a menu of items that includes links to various sub-components of the dashboard system.
- link 402 accesses the “Analysis Center”.
- the analysis center generates custom reports as defined by a particular user.
- the dashboard system is adapted to link to a third party reporting tool, such as the Brio Reporting engine in order to generate these custom reports (See http://www.brio.com, the details of which are hereby incorporated by reference).
- a Link 404 accesses the Site Log Reports which is preferably adapted to link to a third party web log analysis package such as WebTrends.
- the dashboard may incorporate multiple “Log Reports” links, for example, when a client has more than one web-site, or more than one web-log analysis profile for their site.
- a Link 406 shrinks the Dashboard to a smaller version that is better adapted for smaller displays and more conducive to put in a comer of a display where it can remain visible while a user engages in other activities.
- An Exit link 408 closes the dashboard browser window.
- Other links can also be incorporated into the control panel 305 .
- various links to other 3 rd party applications that a user needs to access regularly or links to other company information may be incorporated into the control panel.
- FIGS. 5 - 7 detail the indicators 310 , 325 , and 365 respectively and illustrate the details of each.
- a title bar 420 is positioned at the top of each of the indicators and includes an indicator title 422 , a detail icon 424 , a report icon 426 , and a help icon 428 .
- the Indicator Title 422 shows the logical title of the indicator being displayed.
- the detail icon 424 retrieves a more in-depth display of the information. This may take the form of another grid of indicators e.g., “Today's Stats” indicator, or of a popup window e.g., “HP Load Time” indicator. Details on these popup windows are provided below.
- the report icon 426 (Paper) links to a more in-depth report of the indicator. In many instances this will link to a specific report in the Analysis Center 402 . However, it can also link to other URLs or packages.
- DJIA report icon preferably links to the Yahoo Financial site's DJIA details page.
- the Help Icon 428 (Question-mark) links to a context sensitive help dialog window.
- the help text provides context on the metric, e.g., its definition, how it's calculated, etc.
- a Primary Indicator Number Bar 440 includes a primary indicator value 442 , a unit field 444 , a rate of change field 446 , a directional indicator 448 , a recent change field 450 , and an alert icon 452 .
- the primary indicator value 442 reflects the most current information the indicator has.
- the unit field 444 displays the units of the primary indicator value, such as “pv” which stands for “page views” or “sec” which stands for “seconds.”
- the Rate of Change field 446 indicates the rate of change between the current and the last data point retrieved. For example, the “Today's Stats” indicator in FIG. 6 has a rate of change of 27.7 page views per second.
- the directional indicator 448 indicates the current direction of data displayed in the indicator.
- the Recent change field 450 is the amount that the indicator has changed between the current and last data point. For example, the “HP Load Time” in FIG. 7 has gone up 2.48 seconds in the last hour.
- the recent change field 450 may sometimes reflects a different time period. For example, with the stock market indicators, the number illustrates the change in the indicator since the start of the trading day.
- the alert icon 452 indicates if a data item becomes extremely high or low.
- the alert thresholds are configured on a per-indicator basis. Preferably, alerts can have four levels, from 0 to 3. A level 0 indicator typically does not display an alert icon, but will instead turn the up/down arrow red. Levels 1 through 3 display an icon. If you place your mouse over an alert icon, text will appear and detail the alert.
- a Data Refresh Information bar 470 includes a date field 472 and a refresh interval field 474 .
- the date field 472 indicates the date and time that the currently displayed information represents.
- the refresh interval field 474 describes how frequently the information in this indicator changes.
- the refresh interval comes in three forms: minutes (e.g. 10 m), hours (e.g. 6 h), or days (e.g. 1 d). This is not, however, the amount of time that passes between refreshing the information on the screen. For example, a refresh interval of “1 d” indicates that the information is updated once per day. However, the dashboard system 100 may check for new information every hour so the user receives refreshed information soon after it becomes available.
- An Additional Information bar 480 includes either a Graph field 482 or a table data field 484 .
- the graph field 482 graphs the history of the current indicator or the history of information relevant to the current indicator. Graphs may be displayed as lines, bars, or pie charts.
- the table data field 484 displays either a breakdown of items that make up the main indicator number or additional, less critical statistics that relate to the indicator. For example, a “Registered Users” indicator may detail the different types of users and the number of each.
- FIGS. 8 - 10 show representative examples of detail pop-up windows 500 , 600 , and 700 respectively. Each of the detail popup windows displays a more in-depth view of the data provided by the associated indicator.
- the detail popup windows may come in a variety of formats including, for example, graphs (FIG. 8) and tables (FIGS. 9 and 10).
- FIGS. 8 and 9 are an example of a line graph and its associated table.
- FIG. 10 is an example of a detailed table that does not include an associated graph.
- the detail pop-up window 500 includes a primary indicator number 502 , a data refresh field 504 , a high/low field 506 , a graph/table toggle field 508 , and a graph field 510 .
- the primary indicator number 502 has the same elements that its associated summary Dashboard indicator does, including units, rate of change, up/down arrow, number under arrow, and alert icon.
- the Data Refresh field 504 also displays the same information as on its associated summary Dashboard indicator.
- the High/Low field 506 displays the high and low of the primary indicator number during the time-period in the graph.
- the Graph/Table Toggle field 508 allows a user to toggle between the graph and table versions of the same data.
- the Graph Data field 510 displays the history of the indicator number in either line, bar or pie chart format. Preferably, multiple items can be graphed. For example, on the “HP Load Time” graph in FIG. 8, load times for the home page both with and without ads and a standard “Internet Index” are graphed. Data for a point on the graph will be displayed on mouse-over.
- FIGS. 9 and 10 include a Table Data field 512 that displays the history of the indicator number and possibly the history of related statistics as well in a table form rather than a graph form.
- a dashboard system constructed in accordance with an embodiment of the present invention is preferably adapted to interact with and retrieve data and other business metrics from third party software and web-based applications.
- the VisionPort Dashboard has the ability to extract and display information generated from the third party application CommerceTrends.
- the VisionPort dashboard system nearly every piece of information that is visible in an original CommerceTrends report can be displayed as a VisionPort Dashboard Indicator.
- the following table details some of the standard CommerceTrends reports that may be extracted and displayed by the VisionPort dashboard system: Activity Summary for Report Period Month Visits/Hits Week Visits/Hits Day of the Week Visits/Hits Hour of the Day Visits/Hits Number of Pages Viewed per Visitor Length of Visit by Visits/Views By Kbytes Transferred Over Time of Day Advertising Views and Clicks (summary) Advertising Views (visits/views over time) Advertising Clicks (visits/views over time) Browsers and Systems Top Browsers Microsoft Explorer Browsers Netscape Browsers Top Platforms Errors Technical Statistics Dynamic Pages & Form Errors Client Errors Page Not Found Errors Server Errors Files Top Entry Files Least Requested Entry Files Hits Over Time Most Downloaded Files Most Uploaded Files Top Directories Most Downloaded File Types Hits Over Time Marketing Revenue forecast by Marketing Campaign Campaigns Revenue forecast by Marketing Campaign and Summary Product ROI by Marketing Campaign ROI Percent by Marketing Campaign Pages Top Entry Pages total
- FIGS. 11 and 12 represent an example of a dashboard setup in accordance with an embodiment of the present invention where a “global” association has one dashboard configuration (FIG. 11) and each of any number of association franchises has a second dashboard configuration (FIG. 12).
- FIG. 11 a “global” association has one dashboard configuration
- FIG. 12 a “global” association has a second dashboard configuration
- FIG. 11 figures are compiled for the association as a whole rather than for any one particular franchise, where the franchise dashboard is particularized for one particular outlet, store, or location. Information that is not of particular importance to a franchise is not displayed on the franchise dashboard. The same is true for the association dashboard. Individual dashboards may be set up for each individual franchise in an association.
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Accounting & Taxation (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- Finance (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Economics (AREA)
- Marketing (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Debugging And Monitoring (AREA)
Abstract
Description
- The present invention pertains to systems and methods for measuring, quantifying, and monitoring business performance and operations, and more particularly, to systems and methods for analyzing key business metrics in a business environment and for providing a visual snapshot of the business metrics specified by a user.
- The availability and maintenance of an electronic business system is rapidly becoming the rule rather than the exception in order for a company to compete in today's marketplace. Interactive web-sites allow companies to reach a wide range of audiences and to actively promote and sell their products, services, or other offerings. However, with the integration of an interactive and dynamic web-site or other on-line business system comes the need to provide for continuous monitoring of the performance and utilization parameters of these automated business systems.
- Systems for performing basic statistical analysis of raw business data are known and available in various forms. For instance, the analysis of traditional business performance indicators such as sales volume, profit margins, inventory levels, order placement, and revenue levels are readily calculated from a database compilation of the relevant information. Known systems are adapted to query a database that contains the raw data used to calculate these performance indicators and are adapted to provide the resulting information to a user in various formats. Similarly, known systems and products that merely provide data mining or data warehousing do not allow the flexibility to incorporate analysis systems from third parties or the ability to interrelate the corresponding business metrics with ongoing business operations.
- In the context of an electronic commerce-based business environment, access to dynamic information such as system load time, web-site visits (“page hits”), registered users, page refresh rate, and product ordering demographics is also required in order to effectively monitor and evaluate the performance of a particular electronic business system or web-site. When provided to a user, this information is often needed in a dynamic form in order to accurately portray the business metrics associated with the data. However, traditional statistical analysis systems or database applications are not conducive to monitoring information that is constantly being updated or is by its very nature dynamic. Further, the analysis of electronic business metrics often involves complicated query structures and logic algorithms that are not easily replicated on a static system such as a relational database. Known third party applications that specialize in analyzing a particular type of information are often better suited and more efficient at analyzing and reporting this type of information but known systems are unable to effectively integrate these types of third party analysis tools into a monitoring system structured for a particular company.
- Furthermore, known software applications and other systems fail to provide a user with a business metrics monitoring platform that allows real-time integration with strategic performance indicators while also being able to integrate existing legacy data systems and third party applications. Known systems also fail to provide usable information beyond generic statistics and cannot quantify the impact of specific business initiatives on an overall business objective.
- In one aspect, a system for monitoring business performance indicators in a networked environment, comprises a data source having a predefined format, an agent communicatively coupled to the data source, wherein the agent is configured according to the data source format and wherein the agent is operative to gather data from the data source and translate the data into a first modified format thereby creating modified data. The system also comprises a reaper communicatively coupled to the agent and configured to retrieve the modified data from the agent, a data repository communicatively coupled to the reaper and configured to store the modified data an alert detector communicatively coupled to the data repository and configured to compare the modified data with a first configuration parameter, and a dashboard controller communicatively coupled to the data repository and configured to display the modified data in a format defined by a second configuration parameter.
- In another aspect, a system for monitoring business performance indicators in a networked environment, comprises a data source having a predefined format, an agent communicatively coupled to the data source, wherein the agent is configured according to the data source format and wherein the agent is operative to gather data from the data source and translate the data into a first modified format thereby creating modified data. The system also comprises a reaper communicatively coupled to the agent and configured to retrieve the modified data from the agent, a repository manager communicatively coupled to the reaper, a data repository communicatively coupled to the repository manger, an alert detector communicatively coupled to the repository manager, and a dashboard controller communicatively coupled to the repository manager.
- In another aspect, a method for monitoring a business metric in a networked environment, comprises coupling to a data source having a known format, wherein the data source includes data that represents the business metric, configuring an agent according to the data source format, gathering the data from the data source via the agent, translating the data into a first modified format, storing the modified data in a data repository, comparing the modified data with an alert parameter range, displaying the modified data in a format defined by a second configuration parameter, determining whether the modified data falls within the alert parameter range, and producing an alert if the modified data falls within the alert parameter range.
- In yet another aspect, a system for monitoring a business metric in a networked environment, comprises a processor, a data storage device, and an instruction set residing on the data storage device, wherein the instruction set is configured to perform a method comprising coupling to a data source having a known format, wherein the data source includes data that represents the business metric, configuring an agent according to the data source format, gathering the data from the data source via the agent, translating the data into a first modified format, storing the modified data in a data repository, comparing the modified data with an alert parameter range, displaying the modified data in a format defined by a second configuration parameter, determining whether the modified data falls within the alert parameter range, and producing an alert if the modified data falls within the alert parameter range.
- In a further aspect, a system for monitoring business performance indicators in a networked environment, comprises a collector adapted to communicatively coupled to a data source having a predetermined format, wherein the collector is configured according to the data source format and wherein the collector is operative to gather data from the data source and translate the data into a first modified format thereby creating modified data, a data manager communicatively coupled to the collector and configured to manage the input and output of the modified data between the collector and a data storage device, wherein the data manager is adapted to communicatively couple with an alert device, and a display interface communicatively coupled to the data manager and configured to display the modified data in a format defined by a second configuration parameter.
- As will become apparent to those skilled in the art, numerous other embodiments and aspects of the invention will become evident hereinafter from the following descriptions and claims.
- The drawings illustrate both the design and utility of the preferred embodiments of the present invention, wherein:
- FIG. 1 is an architecture diagram of a dashboard system constructed in accordance with an embodiment of the present invention;
- FIG. 2 is an architecture diagram of a dashboard system constructed in accordance with another embodiment of the present invention;
- FIG. 3 is a screenshot of a dashboard in accordance with an embodiment of the present invention;
- FIGS.4-10 are various details of the dashboard of FIG. 3; and
- FIGS. 11 and 12 are sample dashboards for an organization utilizing an association-franchise relationship.
- FIG. 1 shows a general architectural diagram of a
dashboard system 100 constructed in accordance with an embodiment of the present invention. In a preferred embodiment, thedashboard system 100 is one module of an overall business monitoring and reporting system that is utilized to present a user with an overview of business metrics and other key performance indicators. The dashboard will sometimes be referred to herein as the VisionPort™ Dashboard System. VisionPort™ is a trademark of XOR, Inc. - Generally, the
dashboard system 100 provides a high level executive view of key performance indicators for an eBusiness. Thedashboard system 100 preferably presents a visually oriented snapshot of the status of key business metrics identified by a user. Preferably, thedashboard system 100 is an open and extensible framework of computer software or fixed hardware components designed to enable organizations to build real-time measurement and monitoring solutions for applications and systems. The dashboard can be installed at a user's own facility or maintained at an off-site Application Services Provider for a particular client. Additionally, the various components of thedashboard system 100 maybe spread across different platforms or physical locations. - The
dashboard system 100 is preferably configurable on a per-user or per-client basis, customizable to show each user or client exactly what information they want access to. For instance, a marketing group can be shown one set of informnation such as sales and advertising revenues while a company officer can be shown a broader scope of infonnation about the performance of the company. In addition to being able to gather information from legacy or other existing proprietary company information, thedashboard system 100 is adapted to either link to or direct a user to an appropriate 3rd party application tool to get the required information. Thedashboard system 100 is preferably viewable through readily available internet browser software, such as current versions of Internet Explorer® or Netscape Navigator®. Other devices such as WAP-compatible phones, Palm OS based devices, Blackberry, VoiceXML, and email programs are also preferably compatible with the information provided by thedashboard system 100 so that a user can access the information from a variety of interface devices. In that respect, the framework of thedashboard system 100 is preferably extensible in order to support a wide range of application interfaces. - The diagram depicted in FIG. 1 provides an overview of the technical architecture of a preferred dashboard system framework and its system requirements. The
dashboard system 100 includes four primary components, areaper 120, one ormore agents 140, analert detector 160, and adashboard controller 180. While each of these four primary components will be described in more detail below, the following is an overview of their function and structure. Briefly, theagents 140 are used to gather information from various data sources, third party applications, data servers, legacy systems, etc., and reformat the data into a common scheme using XML. Thereaper 120 is adapted to contact the agents in order to gather the reformatted data and store the reformatted data in a data repository (reference number 124 in FIG. 1). Thereaper 120 is also preferably adapted to perform data aggregation and history creation. Arepository manager 190 is also shown in FIG. 1 that further manages the inflow and outflow of data to and from thedata repository 124. - The
alert detector 160 reads the data from the repository and compares the information to an alert configuration parameter. If an alert is detected, the alert detector initiates an appropriate action such as an SNMP trigger, paging, email, or visual alert. Thedashboard controller 180 acts as the interface between an end-user and the information in the data repository. The dashboard controller requests information from thedata repository 124 and converts the requested information into a format compatible with the user's selected interface. Thedashboard controller 180 is also adapted to read a configuration file in order to determine which information should be delivered to the user interface. - With continuing reference to FIG. 1, the architecture of the
dashboard system 100 is described in more detail. Data is retrieved via theagents 140 from a number ofdifferent data sources 141. Thedata sources 141 can be presented in any number of different formats, including, without limitation, a client-specific data source 142, apre-supported data source 144, adatamart 146, a weblog analysis tool 148, and athird party application 150. A plurality ofagents 140 are utilized, one for each of the data sources being incorporated into thedashboard system 100. Each of theagents 140 are specifically formatted to interact with its associated data source. In a preferred embodiment, the agents are computer software programs written specifically for each of the respective data sources. - The
reaper 120 communicates with each of theagents 140 by a twoway communication link 152 and is adapted to pull the data gathered by the agents on a scheduled basis. The reaper is adapted to utilize, for example, an XML push/pull routine in order to access and retrieve the data from thereaper 120. An agentpolling configuration file 122 defines the necessary polling schedule information for thereaper 120. A datastore ordata repository 124 serves as a data cache and stores the information gathered by thereaper 120. Arepository manager 190, includes one ormore memory caches 192, and further monitors and directs data that is transferred between thedashboard controller 180, thedata repository 124, thereaper 120, and thealert detector 160. - An
alert detector 160 is in two-way communication with therepository manager 190. The alert detector traps threshold conditions that are defined in analert configuration file 162 and compares the threshold conditions to the data stored in thedata repository 124 and processed by therepository manager 190. Thealert detector 160 also sends notifications or alarms to a user based on definitions set in thealert configuration file 162. Notifications and alarms can be in the form of an SNMP trap, email notification, pager notification, visual notification, or any other type of data transfer alert. For instance, if a data value stored in thedata repository 124 is outside of a range defined in the alert configuration file, the alert detector will send an alarm or notification to the user indicating the “out of range” condition. As an example, if thedashboard system 100 is monitoring web-site hits per hour and the user wants to know if this value rises above a number that corresponds to the maximum capacity of the web server, the alert detector will recognize this condition and send an alert to the user. - The
dashboard controller 180 is communicatively coupled to therepository manager 190 and generally functions to format the visualization of the data gathered by theagents 140 and passed through thereaper 120. A unique per-user configuration file 182 is customized for each user of thedashboard system 100 and defines for each user what information from thedashboard system 100 should be displayed to the user on a display device. The display device can be either a conventional PC monitor, a cell phone display, a PDA display (such as a device operating under the PALM or Windows CE operation systems), a pager display, or any other type of data display device. Additionally, information may be translated into a VoiceXML so that information may be accessed on a voice based system. - The
dashboard controller 180 includes anHTML interface 188 that visually formats the data for display on a conventional web browser in the form of anHTML dashboard 184. Thedashboard controller 180 also includes anotherinterface 191 that formats the data for display on another type of dashboard such as one adapted to display or otherwise communicate WML, VoiceXML, or another data protocol. In this sense, it is contemplated that thedashboard controller 180 may include any number of interfaces. Thedashboard controller 180 is preferably adapted to be expandable to incorporate interfaces for future types of dashboard viewing tools. Each of the components of the dashboard system architecture depicted in FIG. 1 are preferably implemented in computer software residing on some form of fixed or otherwise permanent storage system such as a data server or other network of computers. However, thedashboard system 100 may also be implemented in a fixed hardware format such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA). - FIG. 2 is another embodiment of a
dashboard system 101 constructed in accordance with the present invention. Thedashboard system 101 is similar in structure and function to thedashboard system 100 previously described except that a repository manager is not utilized. Instead, thereaper 120 is directly coupled to thedata repository 124 which is in turn coupled to thedashboard controller 180. - Referring again to FIG. 1, the
dashboard system 100 collects data through itsrespective agents 140. Preferably, an agent is a program that fetches a set of information, formats the information into XML, or some other form of open coding language for storage in thedata repository 124. Agents can be created using programming libraries developed from software languages such as PERL and JAVA as well as other programming languages. Eachagent 140 can return one or more data items can be designated to return information in one of two ways. First, each of theagents 140 can return information dynamically. In the thereaper 120 contacts the agent program via a Hyper Text Transfer Protocol (HTTP). Theagent 140 then dynamically gathers the information and delivers XML to thereaper 120. Second, the agent can return information statically. In this manner, the agent is run at a time when the data is known to have changed, such as when a nightly processing routine is completed. Theagent 140 then writes its XML out to a static file. Thereaper 120 is further configured to access this static XML file. Agents can be formatted to get their associated data through a variety of means including SQL queries, text file processing, HTML scraping via an HTTP call, incoming email processing, SNMP, or spreadsheet exports. Other data gathering means are contemplated by the present invention and the above list is not meant to be limiting. Appendix A, which is hereby incorporated by reference, contains a listing of exemplary agents that have been developed for use in accordance with a dashboard system constructed in accordance with an embodiment of the present invention. Appendix A also includes selected information from future agents that are contemplated by a dashboard system constructed in accordance with an embodiment of the present invention. - When an agent generates its data, it preferably converts that data into XML or another open code environment. The XML format is used for communication between the
reaper 120 and theagents 140, and from therepository manager 190 to thedashboard controller 180. - As a typical example, dashboard XML files are structured in the following manner:
<?xml version=“1.0” standalone=“no”?> <!DOCTYPE dashboard SYSTEM “http://www.xor.com/xml/dtds/dashboard.dtd”> <dashboard> . . .individual data items. . . </dashboard> - Dashboard Data Types—Generally, each piece of data assembled by the dashboard must conform to a data type. “Is a” notation is preferably used to declare inheritance between the different data types. When one data type “is a”, it inherits the required and optional attributes from the given data type. The list compiled in Appendix B, which is hereby incorporated by reference, contains examples of data types supported by a dashboard system constructed in accordance with an embodiment of the present invention.
- It is noted that the information in Appendices A and B only represents examples of agents and dashboard data types created for specific applications. It is contemplated that agents and data types may be created for any type of existing data or third party data processing application. Customized libraries can be established for use in creating agents. For example, PERL libraries may be established to supply methods for creating new dashboard objects, “freezing objects into XML”, “thawing” objects into objects, and “selecting” a data item from within an XML structure. An example PERL library is included at Appendix C which is hereby incorporated by reference.
- In addition to the agents described in Appendix A, the following are additional agent descriptions that may be created for a dashboard system constructed in accordance with an embodiment of the present invention. For example, the agents can be adapted to have the ability to “push” data to the reaper, rather than the data being polled. This may be accomplished by having the reaper supply an agent with a “callback” URL. The agent can then hit the callback URL to supply data to the reaper. An optional sequence/version number may be added to the dashboard XML item. Agents that write static HTML files would update the version number each time they write the file. Then, the reaper can use the version number to determine if the dataset has changed. Agents can also be adapted to have the ability to accept arguments from the reaper. For example, a generic “SQL Agent” could be created that accepts a SQL statement from the reaper and returns the results.
- The reaper gathers XML data by contacting the agents. The agents are contacted based on information supplied through a reaper configuration file (agent
polling configuration file 122 from FIG. 1). Preferably, the reaper configuration file supplies agent information for all agents, across all clients or there may exist multiple reaper configuration files; one for each client. The following represents a preferred example of a reaper configuration file:Reaper Configuration File Description Indicates an individual agent that should be contacted to gather information. Notes The name of each agent in the top level, or in a group (see below) must be unique. If an object disappears from the XML tree returned by an agent, it should be removed from the repository. Attributes Name The name of the XML file that the XML tree returned by this agent will be stored in. source_url The URL of the agent that supplies the XML data. Enabled Indicates whether this agent is currently being refreshed (used). Valid values are true and false. Default: true Path The name of the directory that the XML file will be stored in. If not specified, will inherit from its group (in which case it must be set in the group; see below). Refresh The refresh frequency in seconds or a cron-style entry (see man 5 crontab). If not specified, will inherit from its group(in which case it must be set in the group; see below). rate_type Indicates that the reaper should automatically generate a rate field for this agent. The rate is calculated after new data is stored and the history is updated. If not specified, will inherit from its group (if set in the group). Valid values are none, difference: number, percent: number, persecond: number, median: number, averagechange: number, and averagevalue: number. Default: none. Precision Specifies a precision to limit all number elements to. A precision of “1” will make numbers look like “2.5”. Default: leaves numbers as-is. History Indicates that the reaper should accumulate history for items within the XML tree returned by the agent. The history value specifies how many historical values should be kept. Default: 0 - The different rate_types that may be requested in the above configuration file example are further defined as follows:
none Do not populate the rate field. difference: number Rate is calculated as: (current_value − history[number]). For example, difference: 1 calculates the difference between the current and previous values. If number is greater than the size of the history, then the oldest item in the history will be used instead. percentage: number The rate_type is calculated in the same manner as difference, but a percentage change is placed in the rate field instead of the difference. median: number Rate is calculated as: (current_value + history[number])/2. For example, median: 5 calculates the median between the current and 6th history value. persecond: number Rate is calculated as: (current_value − history[number])/ (current_timestamp − timestamp[number]). For example, persecond: 1 calculates the difference between the current and previous in units per second. average: number Calculates the average of the history items, from current to history[number]. Averagechange: number Calculates the average change, from current to history[number]. - History Details—When scanning an XML tree returned by an agent, each element of type number (or subclass thereof) will have history accumulated for it by reading the previous list of values (if any) from the repository and adding the new value to the end of the list. Text and HTML objects will have their history stored by separating chunks of information by a null character. Other object types will simply have the latest value stored in the repository.
- An element will not be pushed onto the history if its current timestamp is the same as the most recent item in the history list. This would occur when the agent program is setting the timestamp explicitly. For example, an agent may choose to set the timestamp to the time that the agent knows the data was last updated (with web logs, that is likely the timestamp of the previous day). If an agent provides its own history, then that history should be used instead, and not be overwritten.
- An element of history length of X actually stores X+1 elements in its history. The “zero” element in the history array is the same as the current value/timestamp.
Elements 1 to X+1 are the historical values/timestamps. - The following is an example of an Agent XML description:
<agent name=“poll” refresh=“20” path=“/vitamins” rate_type=“difference:0” source_url=“http://www.vitamins.com/cgi-bin/dashboard/agents/ poll”/> <agent name=“feedback” refresh=“120” path=“/vitamins” enabled=‘false’ history=‘100’ source url=“http://www.vitamins.com/cgi-bin/dashboard/agents/ feedback”/> - Agents may be classified and categorized by groups. Each group preferably has the following characteristics associated with it.
Description Specifies a group of related agents. Optionally specifies that the reaper should aggregate multiple items together into a single item. Notes The name of each group must be unique. Attributes name A unique name that can be used to refer to this group. Also used as the XML file name where the aggregated numbers will be stored if aggregation is enabled. enabled Indicates whether this group is currently being refreshed (used). Valid values are true and false. If false, then none of the agents in the group will be refreshed, independent of the settings of their individual flags. If true, then each agent can specify its own enabled status. Default: true path The directory location in which to store the resulting aggregated numbers, if aggregation is enabled (see below). If a path is defined here, it will be used as a default for agents in the group that don't have a path defined. refresh The refresh frequency in seconds or a cron-style entry (see man 5 crontab). If specified, it will be used as a default foragents in the group that don't have a refresh defined. precision Specifies a default precision to limit all number elements to in the group. A precision of “1” will make numbers look like “2.5”. Default: leaves numbers as-is. rate_type Indicates that the reaper should automatically generate a rate field for the agents in this group. If specified, it will be used as a default for agents in the group that don't have a rate_type defined, and for aggregation, if it is enabled (see below). Default: none. history Indicates that the reaper should accumulate history for the aggregated items (see below), and how many historical values should be kept. If specified, it will also be used as a default for agents in the group that don't have a history defined. Default: 0 Aggregate Indicates that numeric aggregation should be performed on Numeric the values within this group. Valid values are none, sum, average, and overlay. Default: none Aggregate Indicates that aggregation of other, non-numeric data types Other should be performed on all the values within this group. Valid values are none, merge, and overlay. Default: none Aggregate Indicates the allowable time offset in seconds to use when Maxoffset aggregating history items. Default: infinite Aggregate Indicates that a rate of change should be calculated for the Rate_type aggregated values. See the “rates” section below for possible values. Default: none - Aggregation—When aggregation is enabled in a group, all of the child trees listed in the group will be scanned and a new tree will be constructed that contains elements from both trees. Corresponding elements in the tree (same name and location in the XML tree) are aggregated according to the following rules:
- aggregate_numeric=sum|average
- Items and their subtypes will be numerically summed/averaged. The latest value of each item will enter into the calculation of the aggregate value, regardless of its timestamp. History items will be summed/averaged if their timestamps fall within a given number of seconds of each other. If they do not, then the latest of the timestamped values is used.
- aggregate_numeric=overlay
- Overlays the new information on top of the old. For example, old_xml.overlay(new_xml) would overlay the contents of old_xml with the contents of new_xml. Any elements that are in new_xml that are not in old_xml will also be kept. Anything in the intersection of the two is defaulted to new_xml.
- aggregate_numeric=none
- No numeric aggregation is performed.
- aggregate_other=overlay
- Works the same as aggregate_numeric overlay, but with non-numeric items.
- aggregate_other=merge
- Appends as best possible non-numeric items. See specific data types below for definition.
- aggregate_other=none
- No non-numeric aggregation is performed.
- Details for specific data-types—gauge items behave similarly to number items, with the added feature that the minimum value will be the greatest value of all the minimums of the items being aggregated, but it will be suppressed if not specified for any of the items. Likewise for the maximum (maximum value of all maximum values; suppressed if not specified for any item). “Text” items should have their contents concatenated, separated by newlines, in the order listed in the reaper configuration file. HTML items should have their contents concatenated, separated by an <HR>, in the order listed in the reaper configuration file. “Image” items should not be aggregated, and will generate an error if any are present in the XML trees specified within the group. “Chart” items should be aggregated as follows. The labels should be the union of all label values. The datasets should be the union of all datasets. Elements of datasets with identical names should have the corresponding values of the datasets numerically summed/averaged/overlayed.
- For the purposes of averaging, the actual number of items summed will be used to calculate the average. In other words, if there are 5 agents in the group, but one agent does not return a particular value, then the remaining 4 values will be summed and divided by 4 to calculate the average. If the agent wishes a “zero” value to be included in the average, it should return an item with an explicit value of 0.
- An error will occur if items to be aggregated do not have exactly the same type (e.g. it will be an error to try to aggregate text and image or number and gauge). The individual XML trees retrieved from each agent will be stored as in the repository, so that a UI can retrieve the individual values as well as the aggregate. The historical values will be merged into a single list. Corresponding history items will be summed/averaged/overlayed (e.g.: sum=history—1[0]+history—2[0]). The reaper will do some level of error checking, as defined by aggregate_maxtimeoffset, to make sure that the timestamps are not different by more than this value. If they are, the history item with the newer timestamp is used, and a warning is logged.
- The following is an example of a Group XML example:
<group name=‘stuff’ enabled=‘false’> <agent name=“poll” refresh=“20” path=“/vitamins” source_url=“http://www.vitamins.com/cgi-bin/agents/poll”/> <agent name=“feedback” refresh=“20” path=“/vitamins” enabled=‘false’ source_url=“http://www.vitamins.com/cgi-bin/agents/feedback”/> </group> <group name=‘all_web’ path=‘/vitamins/web’ aggregate_numeric=‘sum’ aggregate_other=‘none’ aggregate_rate_type=‘sum’ aggregate_maxoffset=‘60’> <agent name=“www1” refresh=“60” path=“/vitamins/web” source_url=“http://www1.vitamins.com/cgi-bin/agents/web”/> <agent name=“www2” refresh=“60” path=“/vitamins/web” source_url=“http://www2.vitamins.com/dashboard/web”/> </group> - This above example generates the following XML repository files:
- /vitamins/web/www1.xml: the raw values from www1.vitamins.com
- /vitamins/web/www2.xml: the raw values from www2.vitamins.com
- /vitamins/web/all_web.xml: the aggregated values from both servers
- When creating an aggregate object, the aggregate itself should be locked before it starts to process its group's items. This fixes possible deadlocking and agent data changing in the middle of an aggregation.
- The following additional features of a dashboard reaper constructed in accordance with an embodiment of the present invention are also contemplated. To support an agent “push” enhancement, the reaper may supply the agent with a “callback” URL. The agent can then hit the callback URL to supply data to the reaper. To support an agent “version” enhancement, the reaper may be adapted to recognize the version number to determine if the dataset has changed. To support an agent “argument” enhancement, the reaper may be adapted to support attribute(s) to send arguments to an agent that is designed to be “generic.” Exceptions may be indicated if the reaper is not able to contact an “enabled” agent. For example, an email may be sent to an administrator.
- The reaper may also be adapted so that the reaper children that are performing a request can “time out.” This timeout value would default to something reasonable (i.e. 60 seconds) and may be overwritten by an agent/group specific “timeout” attribute. The controller can be adapted to make a dynamic request for information. If the reaper does not have the request in the datastore, the reaper would then contact the appropriate agent for the information.
- A configuration file for each client may be set up and a configuration file for “generic” information (e.g.: News, Stock Market Quotes) may also be set up. The reaper should be able to be told to re-read a specified configuration file.
- An “alert” can be set on any Dashboard indicator and is triggered when an indicator does, for example, one of the following: 1) descends below a certain threshold; 2) rises above a certain threshold; or 3) is on or between two numbers. When an alert is defined, a severity and an action are associated with it. For example, severity levels of zero through three can be defined (this can be expanded upwards if needed). Actions that can be taken include: 1) Visual: An element in the indicator changes, such as an arrow turning red; 2) Email: An email is sent to a specified address. This address could be a normal user's address, a list of users or a pager email address (most alpha-numeric pagers have associated email addresses); 3) SNMP: An SNMP “trap” can be triggered. SNMP stands for “Simple Network Management Protocol” and is used by network monitoring software to gather information and alerts about routers, switches, machines and other network-enabled devices. For example, SNMP could be used to alert a user's monitoring software. In order for the controller to know if an alert has been triggered, the Alert Detector writes out an XML file that contains the appropriate alert information.
- Preferably, an alert configuration file supplies alert information for all agents, across all clients or there may exist an alert configuration file for each user or client. The following describes the contents of the configuration file that are wrapped with an alerts XML tag.
Action Description Indicates an action to take when an alarm is triggered. Notes The name of each action must be unique. Attributes name The name of the action. This is used in the monitor tags to indicate which action to take when an alert is triggered. Type The type of action to take. Can be one of: display: Indicate that an alert has happened on the dashboard display email: Email a notification to someone that an alert has occurred snmp: Trigger an SNMP trap destination The contents of the destination depends on the action type. display: The alert file to write its alert XML to. email: Email address(es) of the people to notify. Note that this can also be a pager email address snmp: MIB color Only set for actions of type “display”. The color is made available to the templating engine. template Only valid for actions of type “email”. Email template to use to compose the email. subject Only set for actions of type “email”. The subject of the email being sent. Once Only set for actions of type “email”. Indicates if an alert should be sent only once (set to “yes”), or every time that the alert detector uses this action type. snmp Only set for actions of type “snmp”. The MIB address to send an the alert to. XML <action name=”EmailAdmin” type=”email” template=”emailbody” destination=”admin@xor.com” subject=”Alert” once=”yes” /> <action name=”Down” type=”display” color=”#ff0000” destination=”ALERTS” /> <action name=”Neutral” type=”display” destination=”ALERTS” color=”#77ccff” /> <action name=”Up” type=”display” destination=”ALERTS” color=”#66ffcc” /> -
Monitor Description Surrounds a block of level tags and specifies the XML to look at to detect an alert. Notes The name of each monitor must be unique. Attributes name The name of the monitor. Source The name of the source XML file or an alias to the source XML file (specified in data.xml). select A selection of a data item inside the source. Only required if the source doesn't point to the item to look at. XML <monitor name=”DJIA” source=”DJIA” select=”difference/value”> . . . </monitor> -
Level Description Defines the actual alert ceiling/floors for this monitor. A level may only be specified inside a monitor. Notes The name of each level within its monitor must be unique. Attributes name The name of the action. This is used in the monitor tags to indicate which action to take when an alert is triggered. Eff_type Effective start date, specific as YYYYMMDD. If one is not specified, it will begin immediately. Low If both a low and high are set, the low specifies the floor for this alert. If an alert falls on or between the low and high (inclusive), this alert is triggered. If only a low is set, then an alert is triggered if the value falls on or below the low. High If both a low and high are set, the high specifies the ceiling for this alert. If an alert falls on or between the low and high (inclusive), this alert is triggered. If only a high is set, then an alert is triggered if the value falls on or above the high. XML <level name=”down” eff_date=”20010101” low=”−25” high=”0”> . . . </level> -
Result Description Defines tags to set in the template engine for this level. A result can only be specified inside a level. Attributes directional Tag used in the templating engine to indicate part of the name of a directional graphic to display. Msg Message to display to the end-user when they place their mouse over the alert icon. Severity The severity level for this alert. Ranges can be 0 through 3. 0 is typically is visual-only alert (change a directional graphic red, for example), while alerts 1 through 3 typicallyindicate a more severe problem. An alert icon is displayed to the user for severity levels 1 through 3.XML <level name=”down” eff_date=”20010101” low=”−25” high=”0”> <result directional=”red” msg=”Falling” notes=”” severity=”0”/> </level> -
Log Description Defines the action that this alert uses. Attributes action Indicates the action name to trigger for this alert. destination Destination XML file; overwrites the destination specified in the action. XML <level name=“down” eff_date=“20010101” low=“−25” high=“0”> <log action=“Down” destination=“DJIAalert”/> </level> - The following is an example of an XML scheme for a component that represents a DJIA stock market index alert.
<alerts> <action name=“Down” type=“display” color=“#ff0000” destination=“ALERTS”/> <monitor name=“DJIA” source=“DJIA” select=“difference/value”> <level name=“down” eff_date=“20010101” low=“−25” high=“0”> <result directional=“red” msg=“Falling” notes=“” severity=“0”/> <log action=“Down” destination=“DJIAalert”/> </level> <level name=“down25” eff_date=“20010101” low=“−50” high=“−25”> <result directional=“red” msg=“Down over 25” notes=“” severity=“1”/> <log action=“Down” destination=“DJIAalert”/> </level> <level name=“down50” eff_date=“20010101” low=“−75” high=“−50”> <result directional=“red” msg=“Down over 50” notes=“” severity=“2”/> <log action=“Down” destination=“DJIAalert”/> </level> <level name=“down75” eff_date=“20010101” low=“−75”> <result directional=“red” msg=“Down over 75” notes=“” severity=“3”/> <log action=“Down” destination=“DJIAalert”/> </level> </monitor> </alerts> - The above XML alert is set up to behave in the following manner. If the change in the index since the market opened is positive, no alert is set. If the change in the index since the market opened is between 0 and −25,
alert severity 0 is triggered, and the down-arrow is made red. If the change in the index since the market opened is between −25 and −50,alert severity 1 is set, the down-arrow is made red and an “Alert Severity 1” icon is displayed on the indicator. If the user places their mouse over the indicator they will see the text “Down over 25”.Alert severity 2 is set when the change in the index since the market opened is between −50 and −75. The down-arrow is made red and an “Alert Severity 2” icon is displayed on the indicator. If the user places their mouse over the indicator they will see the text “Down over 50”. If the change in the index since the market opened is over −75,alert severity 3 is set, the down-arrow is made red, and an “Alert Severity 3” icon is displayed on the indicator. If the user places their mouse over the indicator they will see the text “Down over 75”. - It is also contemplated that in a dashboard system constructed in accordance with an embodiment of the present invention, the alert detector may have the following additional features. Users can have the ability to specify Key Performance Indicators. The controller gets its information from a user/client-specific output file and there are user/client-specific alarm configuration files. A single alarm file may “include” another. Duplicate alarms would be overwritten by the alarms in the included file. Alarms may activate/deactivate based on a start and end date and time (i.e.: today the ceiling is 10, tomorrow the ceiling is 20). Alarms may be dynamically set by requesting ceiling/floor information from another data source. A user interface may be created for changing alarm information and alerts may be set that span different agents.
- The dashboard controller configuration is composed of a large set of files that can be configured per user. These include HTML/ASCII templates, data location definitions, and the display configuration. The basic configuration is composed of the following directory hierarchy:
dashboard | +-conf | | | +-full Default configuration directory; contains a configuration file for each user | +-templates | +-default | | | +-framed | +-htmltable | +-<other types of rendering templates> | +-<username> Optional; this is only used if the user/client needs a custom template | +-framed +-htmltable +-<other types of rendering templates> - The conf directory contains the XML files that define the screen display. Each user has one configuration file preferably named username.xml. In addition, there is a file that defines aliases that reference the dashboard data repositories. It is preferably called data.xml, but may be changed so that the filename is specified in the username.xml file.
- The templates directory contains a default directory, and may contain a directory hierarchy for each user. The default directory is used if a template file can't be found in the user directories. These directories contain templates, ending in .tmpl, specific for the display renderers. Each type of renderer contains a set of templates that correspond to either the type of dashboard object or a template type specified in the user configuration. The templates are processed to replace a set of tags with the data values from the data sources. There are five tags that are used to do this:
- <TMPL_VAR NAME=“name”>
- This is a basic “name” substitution. If a data element specified by name exists in the user configuration file, then that value will be inserted into the output.
- <TMPL_IF NAME=“name”> . . . </TMPL_IF>
- This is a conditional statement. The text between the beginning and ending TMPL_IF tags will be included in the output if the value of name evaluates to true in the Perl sense: either non zero or a string. This tag also has a <TMPL_ELSE> tag that can be used for if-then-else scenarios.
- <TMPL_UNLESS NAME=“name”> . . . </TMPL_UNLESS>
- This is the reverse of the TMPL_IF tag. It outputs its text if the value of name evaluates to false in the Perl sense: 0 or undefined.
- <TMPL_LOOP NAME=“name”> . . . </TMPL_LOOP>
- This is the basic looping mechanism. The TMPL_VAR and TMPL_IF tags can be contained within the loop. The loop is ran until it has iterated over all of the data values contained in the name (as specified in the user configuration file).
- <TMPL_INCLUDE NAME=“filename.tmpl”>
- Includes the named file in this template. Works just like a server-side include.
- The following is a preferred example of a template:
<TABLE> <TR> <TD>name = <TMPL_VAR NAME=“name”></TD> <TD>Description = <TMPL_VAR NAME=“description”></TD> </TR> <TMPL_LOOP NAME=“load”> <TR> <TD> <TMPL_IF NAME=“machine_name”> <TMPL_VAR NAME=“machine_name”> </TMPL_IF> </TD> <TD> <TMPL_IF NAME=“machine_load”> <TMPL_VAR NAME=“machine_load”> </TMPL_IF> </TD> </TR> </TMPL_LOOP> </TABLE> - The following is a preferred example of an output file:
<TABLE><TR> <TD>name = Load Times</TD> <TD>Description = Load Times on Client Machines</TD> </TR> <TR> <TD> Machine 1: </TD> <TD> 1.5 </TD> </TR> </TR> <TR> <TD> Machine 2: </TD> <TD> 3.2 </TD> </TR> </TABLE> - Data configuration files contain XML entities that define aliases (shortcuts) to the Dashboard data repository items. These aliases are used in the controller configuration files. The default data file is preferably located in a file labeled /info/dashboard/ conf/full/data.xml, but there may be one file per client/user (named the client/user). The aliases add an abstraction layer that may be able to be used to reduce the overall number of templates that are needed.
- The following is a preferred example of a data configuration file:
Load Description Associates an alias name to a data file Notes At a minimum, a load should be created for each XML data file that will be accessed in the user's display configuration. Attributes Name The name of the alias. The name of each load should be unique. File The full filename of the repository data file. In a future release, this should be changed to be relative to the base directory of the dashboard repository. select A path into the specified file's XML data tree. This attribute allows an alias to point to a data node that is deep within the XML tree. If a select is not specified, then the alias simply points to the data file. XML <load name=”servicemetrics” file=”/info/dashboard/repository/servicemetrics/sm.xml”/> <load name=”historical_stats” file=”/info/dashboard/repository/tsn/weblog/historical.xml” select=”sitestats”/> <load name=”weblog” file=”/info/dashboard/repository/tsn/weblog/ weblog.xml”/> <load name=”weblog_box1” file=”/info/dashboard/repository/tsn/weblog/box1.xml” select=”sitestats/www.sportingnews.com/today”/> - Each display configuration file defines the display of a specific user or client. They work with the aliases defined in the data configuration. The configuration file contains the following hierarchy of elements which are described in more detail in Appendix D, which is hereby incorporated by reference:
- navorg - section - construct - item - All of the tags can contain one or more attr elements, which behave similar to an entity/attribute that is included within the main element tag. This is primarily used for readability, when an item has a lot of attributes. For example, a layout attribute can be specified with either:
<section position=“1”>...</section> (or) <section> <attr name=“position”>1</attr> ... </section> - There are a number of variables that are automatically populated for templates. Whether or not a variable is populated depends on the data elements that a template is given and the item types being displayed. A list of variables are found in Appendix E, which is hereby incorporated by reference.
- The templates used by the dashboard controller are accessed by using template=“templatename” in a construct. A number of templates have already been developed and are listed in Appendix F, which is hereby incorporated by reference. These have been developed to provide a high level of flexibility across many different types of metric measurements.
- A dashboard controller utilized in a dashboard system constructed in accordance with an embodiment of the present invention may also be adapted to include the following features. Object caching may be added in order to speed up the display. “Include” files may be added inside the Controller configuration files, so that similar per-user configuration files can be easily made. Included files would overwrite duplicate elements. Display thresholds for KPIs (Key Performance Indicators) and alerts may be graphically displayed and alternate graphing engines may be used.
- The following description provides a preferred embodiment of a dashboard system constructed in accordance with the present invention. The example is representative of an implemented Dashboard system as designed and built by XOR, Inc. of Boulder, Colo. The dashboard is preferably password protected and can be customized for different executives, departments or partners of an eBusiness. Although the embodiment described herein is presented in the form of a HTML internet-based environment, a Wireless Application Protocol (WAP) version of the Dashboard is also contemplated in order to provide dashboard information to a number of different handheld or wireless devices operating on a variety of platforms.
- FIG. 3 represents an exemplary
main dashboard screen 300 of a dashboard system constructed in accordance with an embodiment of the present invention. Preferably, thedashboard screen 300 is broken down into a grid consisting of up to sixteen indicators although more or less indicators may be utilized. As shown in FIG. 3, the sixteen indicators are labeled (and described) as 305 (control panel), 310 (Dow Jones Stock Market Indicator), 315 (NASDAQ stock market ticker), 320 (S&P 500 stock market ticker), 325 (web site statistics), 330 (non-web site statistics), 335 (advertising statistics), 340 (revenue statistics), 345 (previous day web site statistics-yesterday), 350 (previous day web site statistics-week ago), 355 (average web site statistics), 360 (1 month average web site statistics), 365 (home page load time statistics), 370 (average system load time statistics), 375 (web page visits per day), and 380 (registered users). Preferably, a blank box is displayed whenever there are fewer than 16 indicators in use (e.g. when a 3×3 or 3×4 array of indicators are used). Appendix G, which is hereby incorporated by reference, provides a table that describes of each of the above indicators in further detail. Upon opening in a browser-based environment, the dashboard preferably will resize the browser window to the dimensions necessary to display the indicators. - The uppermost
left indicator 305 is referred to as a control panel and is preferably present on all dashboard embodiments. FIG. 4 shows thecontrol panel 305 in further detail. Thecontrol panel 305 contains a menu of items that includes links to various sub-components of the dashboard system. For example, link 402 accesses the “Analysis Center”. The analysis center generates custom reports as defined by a particular user. Preferably, the dashboard system is adapted to link to a third party reporting tool, such as the Brio Reporting engine in order to generate these custom reports (See http://www.brio.com, the details of which are hereby incorporated by reference). - A Link404 accesses the Site Log Reports which is preferably adapted to link to a third party web log analysis package such as WebTrends. The dashboard may incorporate multiple “Log Reports” links, for example, when a client has more than one web-site, or more than one web-log analysis profile for their site. A
Link 406 shrinks the Dashboard to a smaller version that is better adapted for smaller displays and more conducive to put in a comer of a display where it can remain visible while a user engages in other activities. AnExit link 408 closes the dashboard browser window. - Other links can also be incorporated into the
control panel 305. For example, various links to other 3rd party applications that a user needs to access regularly or links to other company information may be incorporated into the control panel. - Each of the indicators detail a different aspect of a user's eBusiness. There are several generic types of indicators, which are detailed below. Indicators with other formats, can be created if desired. As an example, FIGS.5-7 detail the
indicators title bar 420 is positioned at the top of each of the indicators and includes anindicator title 422, adetail icon 424, areport icon 426, and ahelp icon 428. TheIndicator Title 422 shows the logical title of the indicator being displayed. Since the displayed title may be truncated or otherwise abbreviated, a user may utilize thehelp icon 428 to get a full detailed description of what information the indicator is displaying. The detail icon 424 (Magnifying Glass) retrieves a more in-depth display of the information. This may take the form of another grid of indicators e.g., “Today's Stats” indicator, or of a popup window e.g., “HP Load Time” indicator. Details on these popup windows are provided below. The report icon 426 (Paper) links to a more in-depth report of the indicator. In many instances this will link to a specific report in theAnalysis Center 402. However, it can also link to other URLs or packages. For example, the DJIA report icon preferably links to the Yahoo Financial site's DJIA details page. The Help Icon 428 (Question-mark) links to a context sensitive help dialog window. The help text provides context on the metric, e.g., its definition, how it's calculated, etc. - A Primary
Indicator Number Bar 440 includes aprimary indicator value 442, aunit field 444, a rate ofchange field 446, a directional indicator 448, arecent change field 450, and analert icon 452. Theprimary indicator value 442 reflects the most current information the indicator has. Theunit field 444 displays the units of the primary indicator value, such as “pv” which stands for “page views” or “sec” which stands for “seconds.” The Rate ofChange field 446 indicates the rate of change between the current and the last data point retrieved. For example, the “Today's Stats” indicator in FIG. 6 has a rate of change of 27.7 page views per second. This means the client's site was averaging 27.7 page views per second over the last 10 minutes. The directional indicator 448 indicates the current direction of data displayed in the indicator. TheRecent change field 450 is the amount that the indicator has changed between the current and last data point. For example, the “HP Load Time” in FIG. 7 has gone up 2.48 seconds in the last hour. Therecent change field 450 may sometimes reflects a different time period. For example, with the stock market indicators, the number illustrates the change in the indicator since the start of the trading day. Thealert icon 452 indicates if a data item becomes extremely high or low. The alert thresholds are configured on a per-indicator basis. Preferably, alerts can have four levels, from 0 to 3. Alevel 0 indicator typically does not display an alert icon, but will instead turn the up/down arrow red.Levels 1 through 3 display an icon. If you place your mouse over an alert icon, text will appear and detail the alert. - A Data
Refresh Information bar 470 includes adate field 472 and arefresh interval field 474. Thedate field 472 indicates the date and time that the currently displayed information represents. Therefresh interval field 474 describes how frequently the information in this indicator changes. Preferably, the refresh interval comes in three forms: minutes (e.g. 10 m), hours (e.g. 6 h), or days (e.g. 1 d). This is not, however, the amount of time that passes between refreshing the information on the screen. For example, a refresh interval of “1 d” indicates that the information is updated once per day. However, thedashboard system 100 may check for new information every hour so the user receives refreshed information soon after it becomes available. - An
Additional Information bar 480 includes either aGraph field 482 or atable data field 484. Thegraph field 482 graphs the history of the current indicator or the history of information relevant to the current indicator. Graphs may be displayed as lines, bars, or pie charts. Thetable data field 484 displays either a breakdown of items that make up the main indicator number or additional, less critical statistics that relate to the indicator. For example, a “Registered Users” indicator may detail the different types of users and the number of each. - By selecting the detail icon424 a detail pop-up window will be displayed. FIGS. 8-10 show representative examples of detail pop-up
windows - Referring to FIG. 8, the detail pop-up
window 500 includes aprimary indicator number 502, adata refresh field 504, a high/low field 506, a graph/table toggle field 508, and agraph field 510. Theprimary indicator number 502 has the same elements that its associated summary Dashboard indicator does, including units, rate of change, up/down arrow, number under arrow, and alert icon. TheData Refresh field 504 also displays the same information as on its associated summary Dashboard indicator. The High/Low field 506 displays the high and low of the primary indicator number during the time-period in the graph. The Graph/Table Toggle field 508 allows a user to toggle between the graph and table versions of the same data. This field will not appear if only table data is available. TheGraph Data field 510 displays the history of the indicator number in either line, bar or pie chart format. Preferably, multiple items can be graphed. For example, on the “HP Load Time” graph in FIG. 8, load times for the home page both with and without ads and a standard “Internet Index” are graphed. Data for a point on the graph will be displayed on mouse-over. FIGS. 9 and 10 include aTable Data field 512 that displays the history of the indicator number and possibly the history of related statistics as well in a table form rather than a graph form. - In addition to the ability to retrieve proprietary information from a legacy system or another internal corporate database, a dashboard system constructed in accordance with an embodiment of the present invention is preferably adapted to interact with and retrieve data and other business metrics from third party software and web-based applications. For example, the VisionPort Dashboard has the ability to extract and display information generated from the third party application CommerceTrends. In the VisionPort dashboard system, nearly every piece of information that is visible in an original CommerceTrends report can be displayed as a VisionPort Dashboard Indicator. For example, the following table details some of the standard CommerceTrends reports that may be extracted and displayed by the VisionPort dashboard system:
Activity Summary for Report Period Month Visits/Hits Week Visits/Hits Day of the Week Visits/Hits Hour of the Day Visits/Hits Number of Pages Viewed per Visitor Length of Visit by Visits/Views By Kbytes Transferred Over Time of Day Advertising Views and Clicks (summary) Advertising Views (visits/views over time) Advertising Clicks (visits/views over time) Browsers and Systems Top Browsers Microsoft Explorer Browsers Netscape Browsers Top Platforms Errors Technical Statistics Dynamic Pages & Form Errors Client Errors Page Not Found Errors Server Errors Files Top Entry Files Least Requested Entry Files Hits Over Time Most Downloaded Files Most Uploaded Files Top Directories Most Downloaded File Types Hits Over Time Marketing Revenue forecast by Marketing Campaign Campaigns Revenue forecast by Marketing Campaign and Summary Product ROI by Marketing Campaign ROI Percent by Marketing Campaign Pages Top Entry Pages totals and over time Least requested Entry Pages totals and over time Top Exit Pages totals and over time Top Content Groups Single Access Pages Dynamic Pages and Forms Views Over Time Parameter Analysis by Visits/Hits Paths Top Paths Through Site Top Destination Paths Through Site Product Revenue Forecast by Product and Visitor Type Summary (qualified/non-qualified) Revenue Forecast by Product and Referrer Referrers Top Referring Sites Top Referring URLs Top Search Engines Top Search Phrases Top Search Keywords Server By Kbytes Transferred Cluster Load By Hits Balance Visitors New vs. Returning Visitors (Totals and Over Time) Top Authenticated Visitors (Visits/Hits with Over Time) Top Visitors (Visits/Hits with over time) Visits Over Time Number of Visits - FIGS. 11 and 12 represent an example of a dashboard setup in accordance with an embodiment of the present invention where a “global” association has one dashboard configuration (FIG. 11) and each of any number of association franchises has a second dashboard configuration (FIG. 12). In the association dashboard, figures are compiled for the association as a whole rather than for any one particular franchise, where the franchise dashboard is particularized for one particular outlet, store, or location. Information that is not of particular importance to a franchise is not displayed on the franchise dashboard. The same is true for the association dashboard. Individual dashboards may be set up for each individual franchise in an association.
- Although the present invention has been described and illustrated in the above description and drawings, it is understood that this description is by example only and that numerous changes and modifications can be made by those skilled in the art without departing from the true spirit and scope of the invention. The invention, therefore, is not to be restricted, except by the following claims and their equivalents.
Claims (47)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/877,414 US7188169B2 (en) | 2001-06-08 | 2001-06-08 | System and method for monitoring key performance indicators in a business |
PCT/US2002/018069 WO2002102097A1 (en) | 2001-06-08 | 2002-06-07 | System and method for monitoring key performance indicators in a business |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/877,414 US7188169B2 (en) | 2001-06-08 | 2001-06-08 | System and method for monitoring key performance indicators in a business |
Publications (2)
Publication Number | Publication Date |
---|---|
US20030110249A1 true US20030110249A1 (en) | 2003-06-12 |
US7188169B2 US7188169B2 (en) | 2007-03-06 |
Family
ID=25369914
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/877,414 Expired - Lifetime US7188169B2 (en) | 2001-06-08 | 2001-06-08 | System and method for monitoring key performance indicators in a business |
Country Status (2)
Country | Link |
---|---|
US (1) | US7188169B2 (en) |
WO (1) | WO2002102097A1 (en) |
Cited By (98)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030200486A1 (en) * | 2002-04-01 | 2003-10-23 | Navjot Marwaha | System and method for providing common event format using alert index |
US20040034716A1 (en) * | 2002-08-19 | 2004-02-19 | Siemens Westinghouse Power Corporation | Dynamic file-based routing in a broadband communications system |
WO2004017243A1 (en) * | 2002-08-15 | 2004-02-26 | Viscom Limited | A data collection, transfer and visualisation network |
US20040122940A1 (en) * | 2002-12-20 | 2004-06-24 | Gibson Edward S. | Method for monitoring applications in a network which does not natively support monitoring |
US20040128229A1 (en) * | 2002-12-30 | 2004-07-01 | Fannie Mae | System and method for processing data pertaining to financial assets |
US20040153329A1 (en) * | 2003-02-03 | 2004-08-05 | Fabio Casati | System and method for monitoring event based systems |
US20040181685A1 (en) * | 2002-04-01 | 2004-09-16 | Navjot Marwaha | System and method for handling distribution of alerts |
US20040215555A1 (en) * | 2002-12-30 | 2004-10-28 | Fannie Mae | System and method for creating and tracking agreements for selling loans to a secondary market purchaser |
US20040215554A1 (en) * | 2002-12-30 | 2004-10-28 | Fannie Mae | System and method for verifying loan data at delivery |
US20040220873A1 (en) * | 2002-12-30 | 2004-11-04 | Fannie Mae | System and method for defining loan products |
US20040225597A1 (en) * | 2002-12-30 | 2004-11-11 | Fannie Mae | System and method for processing data pertaining to financial assets |
US20040225594A1 (en) * | 2002-12-30 | 2004-11-11 | Fannie Mae | System and method for pricing loans in the secondary mortgage market |
US20040260591A1 (en) * | 2003-06-17 | 2004-12-23 | Oracle International Corporation | Business process change administration |
US20050065941A1 (en) * | 2003-09-23 | 2005-03-24 | Deangelis Stephen F. | Systems for optimizing business processes, complying with regulations, and identifying threat and vulnerabilty risks for an enterprise |
US20050065807A1 (en) * | 2003-09-23 | 2005-03-24 | Deangelis Stephen F. | Systems and methods for optimizing business processes, complying with regulations, and identifying threat and vulnerabilty risks for an enterprise |
US20050102226A1 (en) * | 2002-12-30 | 2005-05-12 | Dror Oppenheimer | System and method of accounting for mortgage related transactions |
US20050102229A1 (en) * | 2002-12-30 | 2005-05-12 | Kemper John L. | Internet-enabled interface for a loan commitment system |
US20050114510A1 (en) * | 2003-03-04 | 2005-05-26 | Error Brett M. | Assigning value to elements contributing to business success |
US20050132048A1 (en) * | 2003-12-12 | 2005-06-16 | International Business Machines Corporation | Role-based views access to a workflow weblog |
US20050131750A1 (en) * | 2003-12-12 | 2005-06-16 | International Business Machines Corporation | Method for tracking the status of a workflow using weblogs |
US20050198021A1 (en) * | 2003-12-12 | 2005-09-08 | International Business Machines Corporation | Visualization of attributes of workflow weblogs |
US20050210052A1 (en) * | 2004-03-17 | 2005-09-22 | Aldridge Gregory E | System and method for transforming and using content in other systems |
US20050216831A1 (en) * | 2004-03-29 | 2005-09-29 | Grzegorz Guzik | Key performance indicator system and method |
US20060059026A1 (en) * | 2004-08-24 | 2006-03-16 | Oracle International Corporation | Compliance workbench |
US20060089889A1 (en) * | 2004-10-22 | 2006-04-27 | 1St Cfo Llc | System for visualizing financial data |
US20060089861A1 (en) * | 2004-10-22 | 2006-04-27 | Oracle International Corporation | Survey based risk assessment for processes, entities and enterprise |
US20060095282A1 (en) * | 2004-10-29 | 2006-05-04 | International Business Machines Corporation | Method and system for displaying prioritization of metric values |
US20060161471A1 (en) * | 2005-01-19 | 2006-07-20 | Microsoft Corporation | System and method for multi-dimensional average-weighted banding status and scoring |
US20060198208A1 (en) * | 2005-03-07 | 2006-09-07 | Lantronix, Inc. | Publicasting systems and methods |
US20060277212A1 (en) * | 2005-06-03 | 2006-12-07 | Error Brett M | User interface providing summary information or a status pane in a web analytics tool |
US20060277198A1 (en) * | 2005-06-03 | 2006-12-07 | Error Brett M | One-click segmentation definition |
US20060277211A1 (en) * | 2005-06-03 | 2006-12-07 | Error Brett M | Incrementally adding segmentation criteria to a data set |
US20070016520A1 (en) * | 2002-12-30 | 2007-01-18 | Gang John E | System and method for facilitating sale of a loan to a secondary market purchaser |
US20070094380A1 (en) * | 2005-10-20 | 2007-04-26 | Samuel Fu | Method and apparatus for collocating application monitoring reports with web applications |
US20070143306A1 (en) * | 2005-12-15 | 2007-06-21 | Jae-Hyuk Yang | Integrated website management system and management method thereof |
US20070143161A1 (en) * | 2005-12-21 | 2007-06-21 | Microsoft Corporation | Application independent rendering of scorecard metrics |
US20070239660A1 (en) * | 2006-03-30 | 2007-10-11 | Microsoft Corporation | Definition and instantiation of metric based business logic reports |
US20070239573A1 (en) * | 2006-03-30 | 2007-10-11 | Microsoft Corporation | Automated generation of dashboards for scorecard metrics and subordinate reporting |
US20070265863A1 (en) * | 2006-04-27 | 2007-11-15 | Microsoft Corporation | Multidimensional scorecard header definition |
US20070282824A1 (en) * | 2006-05-31 | 2007-12-06 | Ellingsworth Martin E | Method and system for classifying documents |
US20080077656A1 (en) * | 2002-09-06 | 2008-03-27 | Oracle International Corporation | Method and apparatus for a report cache in a near real-time business intelligence system |
US20080120211A1 (en) * | 2002-12-30 | 2008-05-22 | Dror Oppenheimer | System and method for modifying attribute data pertaining to financial assets in a data processing system |
US20080148289A1 (en) * | 2002-09-06 | 2008-06-19 | Oracle International Corporation | Method and apparatus for distributed rule evaluation in a near real-time business intelligence system |
US20080162204A1 (en) * | 2006-12-28 | 2008-07-03 | Kaiser John J | Tracking and management of logistical processes |
US20080172414A1 (en) * | 2007-01-17 | 2008-07-17 | Microsoft Corporation | Business Objects as a Service |
US20080172629A1 (en) * | 2007-01-17 | 2008-07-17 | Microsoft Corporation | Geometric Performance Metric Data Rendering |
US20080178148A1 (en) * | 2007-01-19 | 2008-07-24 | International Business Machines Corporation | Business performance bookmarks |
US20080184099A1 (en) * | 2007-01-26 | 2008-07-31 | Microsoft Corporation | Data-Driven Presentation Generation |
US20080183519A1 (en) * | 2006-08-03 | 2008-07-31 | Oracle International Corporation | Business process for ultra vires transactions |
US20080183564A1 (en) * | 2007-01-30 | 2008-07-31 | Microsoft Corporation | Untethered Interaction With Aggregated Metrics |
US20080184130A1 (en) * | 2007-01-30 | 2008-07-31 | Microsoft Corporation | Service Architecture Based Metric Views |
US20080189632A1 (en) * | 2007-02-02 | 2008-08-07 | Microsoft Corporation | Severity Assessment For Performance Metrics Using Quantitative Model |
US7437463B1 (en) * | 2002-06-21 | 2008-10-14 | Polycom, Inc. | Method and means for providing scheduling for a videoconferencing network in a manner to ensure bandwidth |
US20080301444A1 (en) * | 2005-12-07 | 2008-12-04 | Electronics & Telecommunications Research Institut | Apparatus and Method for Providing Personal Information Sharing Service Using Signed Callback Url Message |
US20090112663A1 (en) * | 2006-01-30 | 2009-04-30 | Jay William Benayon | Method and apparatus for business process transformation wizard |
US20090132304A1 (en) * | 2007-11-16 | 2009-05-21 | Sap Ag | Determining a Value for an Indicator |
US20090183104A1 (en) * | 2008-01-03 | 2009-07-16 | Dotson Gerald A | Multi-mode viewer control for viewing and managing groups of statistics |
US20090228485A1 (en) * | 2008-03-07 | 2009-09-10 | Microsoft Corporation | Navigation across datasets from multiple data sources based on a common reference dimension |
US7593893B1 (en) | 2000-06-13 | 2009-09-22 | Fannie Mae | Computerized systems and methods for facilitating the flow of capital through the housing finance industry |
US7653592B1 (en) | 2003-12-01 | 2010-01-26 | Fannie Mae | System and method for processing a loan |
US7657475B1 (en) | 2003-12-31 | 2010-02-02 | Fannie Mae | Property investment rating system and method |
US20100073160A1 (en) * | 2008-09-25 | 2010-03-25 | Microsoft Corporation | Alerting users using a multiple state status icon |
US20100082702A1 (en) * | 2008-09-29 | 2010-04-01 | Honeywell International Inc. | Dynamic vehicle information management |
US7702580B1 (en) | 2000-06-13 | 2010-04-20 | Fannie Mae | System and method for mortgage loan pricing, sale and funding |
US7747526B1 (en) | 2006-03-27 | 2010-06-29 | Fannie Mae | System and method for transferring mortgage loan servicing rights |
US7765151B1 (en) | 2000-06-13 | 2010-07-27 | Fannie Mae | Computerized systems and methods for facilitating the flow of capital through the housing finance industry |
US7801809B1 (en) | 2005-06-24 | 2010-09-21 | Fannie Mae | System and method for management of delegated real estate project reviews |
US7822680B1 (en) | 2003-12-31 | 2010-10-26 | Fannie Mae | System and method for managing data pertaining to a plurality of financial assets for multifamily and housing developments |
US20110004627A1 (en) * | 2009-07-01 | 2011-01-06 | Oracle International Corporation | Dashboard for business process management system |
US7885889B2 (en) | 2002-12-30 | 2011-02-08 | Fannie Mae | System and method for processing data pertaining to financial assets |
US20110119107A1 (en) * | 2006-01-05 | 2011-05-19 | Oracle International Corporation | Audit planning |
US8046298B1 (en) | 2003-07-21 | 2011-10-25 | Fannie Mae | Systems and methods for facilitating the flow of capital through the housing finance industry |
US8165993B2 (en) * | 2002-09-06 | 2012-04-24 | Oracle International Corporation | Business intelligence system with interface that provides for immediate user action |
US8190992B2 (en) | 2006-04-21 | 2012-05-29 | Microsoft Corporation | Grouping and display of logically defined reports |
US8214846B1 (en) * | 2007-09-12 | 2012-07-03 | Sprint Communications Company L. P. | Method and system for threshold management |
US20120191831A1 (en) * | 2011-01-26 | 2012-07-26 | Carl Kanzabedian | System and method for cataloging assets in a network |
US8255454B2 (en) | 2002-09-06 | 2012-08-28 | Oracle International Corporation | Method and apparatus for a multiplexed active data window in a near real-time business intelligence system |
US8261181B2 (en) | 2006-03-30 | 2012-09-04 | Microsoft Corporation | Multidimensional metrics-based annotation |
US20120239675A1 (en) * | 2003-03-04 | 2012-09-20 | Error Brett M | Associating Website Clicks with Links on a Web Page |
US20130013370A1 (en) * | 2008-12-30 | 2013-01-10 | Infosys Limited | System and method for automatically generating an optimized business process design |
US8402095B2 (en) | 2002-09-16 | 2013-03-19 | Oracle International Corporation | Apparatus and method for instant messaging collaboration |
US8495663B2 (en) | 2007-02-02 | 2013-07-23 | Microsoft Corporation | Real time collaboration using embedded data visualizations |
US20130246129A1 (en) * | 2012-03-19 | 2013-09-19 | International Business Machines Corporation | Discovery and realization of business measurement concepts |
US8566693B2 (en) | 2002-09-06 | 2013-10-22 | Oracle International Corporation | Application-specific personalization for data display |
US20130304910A1 (en) * | 2011-01-27 | 2013-11-14 | Gmarkets Inc. | Method and system for generating and providing data alerts |
US8666879B1 (en) | 2002-12-30 | 2014-03-04 | Fannie Mae | Method and system for pricing forward commitments for mortgage loans and for buying committed loans |
US8793236B2 (en) | 2012-11-01 | 2014-07-29 | Adobe Systems Incorporated | Method and apparatus using historical influence for success attribution in network site activity |
US20140334326A1 (en) * | 2011-12-26 | 2014-11-13 | Huawei Technologies Co., Ltd. | Method, Device, and System for Monitoring Quality of Internet Access Service of Mobile Terminal |
US20150066567A1 (en) * | 2013-08-30 | 2015-03-05 | Sap Ag | News mining for enterprise resource planning |
US9092282B1 (en) | 2012-08-14 | 2015-07-28 | Sprint Communications Company L.P. | Channel optimization in a messaging-middleware environment |
US9264338B1 (en) | 2013-04-08 | 2016-02-16 | Sprint Communications Company L.P. | Detecting upset conditions in application instances |
US10402296B2 (en) * | 2014-08-27 | 2019-09-03 | Kabushiki Kaisha Toshiba | Monitor control system and data collection apparatus |
US10521601B2 (en) | 2014-04-30 | 2019-12-31 | Sailpoint Technologies, Israel Ltd. | System and method for data governance |
US11308186B1 (en) | 2021-03-19 | 2022-04-19 | Sailpoint Technologies, Inc. | Systems and methods for data correlation and artifact matching in identity management artificial intelligence systems |
US11461677B2 (en) | 2020-03-10 | 2022-10-04 | Sailpoint Technologies, Inc. | Systems and methods for data correlation and artifact matching in identity management artificial intelligence systems |
US11475380B2 (en) * | 2018-05-14 | 2022-10-18 | Horiba, Ltd. | Vehicle test facility operation rate analysis system and method |
US20230153217A1 (en) * | 2017-03-20 | 2023-05-18 | Netapp Inc. | Methods and systems for managing networked storage system resources |
US12298873B2 (en) * | 2022-11-14 | 2025-05-13 | Netapp, Inc. | Methods and systems for managing networked storage system resources |
Families Citing this family (79)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050086584A1 (en) * | 2001-07-09 | 2005-04-21 | Microsoft Corporation | XSL transform |
DE10159028A1 (en) * | 2001-11-26 | 2003-06-12 | Kenan Tur | Method and arrangement for information processing in monitoring systems for ethics, risk and / or value management as well as a corresponding computer program product and a corresponding storage medium |
US20040049473A1 (en) * | 2002-09-05 | 2004-03-11 | David John Gower | Information analytics systems and methods |
US7529842B2 (en) * | 2002-12-17 | 2009-05-05 | International Business Machines Corporation | Method, system and program product for detecting an operational risk of a node |
US7729931B1 (en) * | 2003-03-17 | 2010-06-01 | Verizon Laboratories Inc. | Systems and methods for comparing and improving sales performance over heterogeneous geographical sales regions |
US7899693B2 (en) * | 2003-06-17 | 2011-03-01 | Oracle International Corporation | Audit management workbench |
US8296167B2 (en) * | 2003-06-17 | 2012-10-23 | Nigel King | Process certification management |
US7941353B2 (en) | 2003-06-17 | 2011-05-10 | Oracle International Corporation | Impacted financial statements |
US8005709B2 (en) * | 2003-06-17 | 2011-08-23 | Oracle International Corporation | Continuous audit process control objectives |
US20040260628A1 (en) * | 2003-06-17 | 2004-12-23 | Oracle International Corporation | Hosted audit service |
US7523401B1 (en) | 2003-09-03 | 2009-04-21 | Theoris Software, Llc | System and method for providing a browser-based user interface |
US7702532B2 (en) * | 2003-12-12 | 2010-04-20 | At&T Intellectual Property, I, L.P. | Method, system and storage medium for utilizing training roadmaps in a call center |
US20050144022A1 (en) * | 2003-12-29 | 2005-06-30 | Evans Lori M. | Web-based system, method, apparatus and software to manage performance securely across an extended enterprise and between entities |
US20050209899A1 (en) * | 2004-03-16 | 2005-09-22 | Oracle International Corporation | Segregation of duties reporting |
US20050256752A1 (en) * | 2004-05-12 | 2005-11-17 | Bala Ramachandran | Method for managing and controlling stability in business activity monitoring and management systems |
US8533027B1 (en) * | 2004-07-30 | 2013-09-10 | Hewlett-Packard Development Company, L.P. | User interface enabling approval/disappoval of revised contract performance indicators |
US20060036595A1 (en) * | 2004-08-12 | 2006-02-16 | International Business Machines Corporation | Role-based dynamically customizable dashboards |
US20060074739A1 (en) * | 2004-09-20 | 2006-04-06 | Oracle International Corporation | Identifying risks in conflicting duties |
US20060074809A1 (en) * | 2004-09-24 | 2006-04-06 | Meyer Barry D | Usage data collection system and method |
US7720751B2 (en) * | 2004-10-12 | 2010-05-18 | Natural Decision Systems, Inc. | System and method of continuous assurance for internal control |
US7676427B1 (en) * | 2004-10-12 | 2010-03-09 | Natural Decision Systems, Inc. | System and method of continuous assurance |
US7827499B2 (en) * | 2004-10-29 | 2010-11-02 | Hewlett-Packard Development Company, L.P. | Hierarchical dataset dashboard view |
US7657842B2 (en) * | 2004-11-12 | 2010-02-02 | Microsoft Corporation | Sidebar tile free-arrangement |
US20060106686A1 (en) * | 2004-11-12 | 2006-05-18 | Oracle International Corporation | Audit procedures and audit steps |
US7523053B2 (en) * | 2005-04-25 | 2009-04-21 | Oracle International Corporation | Internal audit operations for Sarbanes Oxley compliance |
US20070027760A1 (en) * | 2005-07-29 | 2007-02-01 | Collins Robert J | System and method for creating and providing a user interface for displaying advertiser defined groups of advertisement campaign information |
US7840438B2 (en) * | 2005-07-29 | 2010-11-23 | Yahoo! Inc. | System and method for discounting of historical click through data for multiple versions of an advertisement |
US20070033129A1 (en) * | 2005-08-02 | 2007-02-08 | Coates Frank J | Automated system and method for monitoring, alerting and confirming resolution of critical business and regulatory metrics |
US7519694B1 (en) * | 2005-08-24 | 2009-04-14 | Sun Microsystems, Inc. | Method and a system to dynamically update/reload agent configuration data |
US8601383B2 (en) | 2005-09-09 | 2013-12-03 | Microsoft Corporation | User interface for creating a spreadsheet data summary table |
US8095866B2 (en) | 2005-09-09 | 2012-01-10 | Microsoft Corporation | Filtering user interface for a data summary table |
US7627812B2 (en) | 2005-10-27 | 2009-12-01 | Microsoft Corporation | Variable formatting of cells |
US20070174228A1 (en) * | 2006-01-17 | 2007-07-26 | Microsoft Corporation | Graphical representation of key performance indicators |
US20070185746A1 (en) * | 2006-01-24 | 2007-08-09 | Chieu Trieu C | Intelligent event adaptation mechanism for business performance monitoring |
US7770100B2 (en) * | 2006-02-27 | 2010-08-03 | Microsoft Corporation | Dynamic thresholds for conditional formats |
US20070239871A1 (en) * | 2006-04-11 | 2007-10-11 | Mike Kaskie | System and method for transitioning to new data services |
US7912865B2 (en) | 2006-09-26 | 2011-03-22 | Experian Marketing Solutions, Inc. | System and method for linking multiple entities in a business database |
US20080086359A1 (en) * | 2006-10-04 | 2008-04-10 | Holton Peter R | Sales opportunity explorer |
US8606666B1 (en) | 2007-01-31 | 2013-12-10 | Experian Information Solutions, Inc. | System and method for providing an aggregation tool |
US8285656B1 (en) | 2007-03-30 | 2012-10-09 | Consumerinfo.Com, Inc. | Systems and methods for data verification |
US8533661B2 (en) | 2007-04-27 | 2013-09-10 | Dell Products, Lp | System and method for automated on-demand creation of a customized software application |
WO2010004654A1 (en) * | 2008-07-11 | 2010-01-14 | 株式会社 C・B・H | Cost standard determining system for calculating commitment fee |
US9256904B1 (en) | 2008-08-14 | 2016-02-09 | Experian Information Solutions, Inc. | Multi-bureau credit file freeze and unfreeze |
US8055375B2 (en) * | 2008-09-30 | 2011-11-08 | Rockwell Automation Technologies, Inc. | Analytical generator of key performance indicators for pivoting on metrics for comprehensive visualizations |
US20100082292A1 (en) * | 2008-09-30 | 2010-04-01 | Rockwell Automation Technologies, Inc. | Analytical generator of key performance indicators for pivoting on metrics for comprehensive visualizations |
US9251491B2 (en) * | 2009-11-30 | 2016-02-02 | International Business Machines Corporation | Performance-aware enterprise components |
US9652802B1 (en) | 2010-03-24 | 2017-05-16 | Consumerinfo.Com, Inc. | Indirect monitoring and reporting of a user's credit data |
US8639616B1 (en) | 2010-10-01 | 2014-01-28 | Experian Information Solutions, Inc. | Business to contact linkage system |
US8930262B1 (en) | 2010-11-02 | 2015-01-06 | Experian Technology Ltd. | Systems and methods of assisted strategy design |
US9147042B1 (en) | 2010-11-22 | 2015-09-29 | Experian Information Solutions, Inc. | Systems and methods for data verification |
US20120253891A1 (en) * | 2011-04-01 | 2012-10-04 | The Corporate Executive Board | Computer-Implemented Generation Of Roadmap Visualizations |
US9558519B1 (en) | 2011-04-29 | 2017-01-31 | Consumerinfo.Com, Inc. | Exposing reporting cycle information |
US8504912B2 (en) * | 2011-07-15 | 2013-08-06 | Neodata Group S.R.L. | System to forecast performance of online news articles to suggest the optimal homepage layout to maximize article readership and readers stickiness |
US9021308B2 (en) | 2011-08-09 | 2015-04-28 | International Business Machines Corporation | Analyzing a process of software defects handling using percentile-based metrics |
US8533022B2 (en) * | 2011-09-13 | 2013-09-10 | Nandakumar Krishnan Nair | Enterprise wide value chain management system (EVCM) for tracking, analyzing and improving organizational value chain performance and disruptions utilizing corrective actions |
US9318078B2 (en) | 2011-10-31 | 2016-04-19 | Invensys Systems, Inc. | Intelligent memory management system and method for visualization of information |
US9710282B2 (en) | 2011-12-21 | 2017-07-18 | Dell Products, Lp | System to automate development of system integration application programs and method therefor |
US8943076B2 (en) | 2012-02-06 | 2015-01-27 | Dell Products, Lp | System to automate mapping of variables between business process applications and method therefor |
US8805716B2 (en) * | 2012-03-19 | 2014-08-12 | Dell Products, Lp | Dashboard system and method for identifying and monitoring process errors and throughput of integration software |
US8782103B2 (en) | 2012-04-13 | 2014-07-15 | Dell Products, Lp | Monitoring system for optimizing integrated business processes to work flow |
US9606995B2 (en) | 2012-04-30 | 2017-03-28 | Dell Products, Lp | Cloud based master data management system with remote data store and method therefor |
US9158782B2 (en) | 2012-04-30 | 2015-10-13 | Dell Products, Lp | Cloud based master data management system with configuration advisor and method therefore |
US9015106B2 (en) | 2012-04-30 | 2015-04-21 | Dell Products, Lp | Cloud based master data management system and method therefor |
US8589207B1 (en) | 2012-05-15 | 2013-11-19 | Dell Products, Lp | System and method for determining and visually predicting at-risk integrated processes based on age and activity |
US9069898B2 (en) | 2012-05-31 | 2015-06-30 | Dell Products, Lp | System for providing regression testing of an integrated process development system and method therefor |
US9092244B2 (en) | 2012-06-07 | 2015-07-28 | Dell Products, Lp | System for developing custom data transformations for system integration application programs |
US10255598B1 (en) | 2012-12-06 | 2019-04-09 | Consumerinfo.Com, Inc. | Credit card account data extraction |
US20140244343A1 (en) * | 2013-02-22 | 2014-08-28 | Bank Of America Corporation | Metric management tool for determining organizational health |
US9697263B1 (en) | 2013-03-04 | 2017-07-04 | Experian Information Solutions, Inc. | Consumer data request fulfillment system |
US9646276B2 (en) * | 2013-08-23 | 2017-05-09 | AppDynamics, Inc. | Dashboard for dynamic display of distributed transaction data |
US20150127431A1 (en) * | 2013-11-05 | 2015-05-07 | Wal-Mart Stores, Inc. | Performance Evaluation System for Stores |
US10262362B1 (en) | 2014-02-14 | 2019-04-16 | Experian Information Solutions, Inc. | Automatic generation of code for attributes |
US10757154B1 (en) | 2015-11-24 | 2020-08-25 | Experian Information Solutions, Inc. | Real-time event-based notification system |
EP3226184A1 (en) | 2016-03-30 | 2017-10-04 | Tata Consultancy Services Limited | Systems and methods for determining and rectifying events in processes |
US10498858B2 (en) | 2016-12-14 | 2019-12-03 | Dell Products, Lp | System and method for automated on-demand creation of and execution of a customized data integration software application |
CN110383319B (en) | 2017-01-31 | 2023-05-26 | 益百利信息解决方案公司 | Large scale heterogeneous data ingestion and user resolution |
US20180253677A1 (en) * | 2017-03-01 | 2018-09-06 | Gregory James Foster | Method for Performing Dynamic Data Analytics |
US10735183B1 (en) | 2017-06-30 | 2020-08-04 | Experian Information Solutions, Inc. | Symmetric encryption for private smart contracts among multiple parties in a private peer-to-peer network |
US11620403B2 (en) | 2019-01-11 | 2023-04-04 | Experian Information Solutions, Inc. | Systems and methods for secure data aggregation and computation |
Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5781703A (en) * | 1996-09-06 | 1998-07-14 | Candle Distributed Solutions, Inc. | Intelligent remote agent for computer performance monitoring |
US5905868A (en) * | 1997-07-22 | 1999-05-18 | Ncr Corporation | Client/server distribution of performance monitoring data |
US5995943A (en) * | 1996-04-01 | 1999-11-30 | Sabre Inc. | Information aggregation and synthesization system |
US6041362A (en) * | 1995-10-20 | 2000-03-21 | Electronics Data Systems Corporation | Method and system for integrating disparate information technology applications and platforms across an enterprise |
US6055493A (en) * | 1997-01-29 | 2000-04-25 | Infovista S.A. | Performance measurement and service quality monitoring system and process for an information system |
US6108686A (en) * | 1998-03-02 | 2000-08-22 | Williams, Jr.; Henry R. | Agent-based on-line information retrieval and viewing system |
US6119103A (en) * | 1997-05-27 | 2000-09-12 | Visa International Service Association | Financial risk prediction systems and methods therefor |
US6148335A (en) * | 1997-11-25 | 2000-11-14 | International Business Machines Corporation | Performance/capacity management framework over many servers |
US6240295B1 (en) * | 1993-07-20 | 2001-05-29 | @Track Communications, Inc. | Data messaging in a communications network using a feature request |
US6292830B1 (en) * | 1997-08-08 | 2001-09-18 | Iterations Llc | System for optimizing interaction among agents acting on multiple levels |
US6349290B1 (en) * | 1998-06-30 | 2002-02-19 | Citibank, N.A. | Automated system and method for customized and personalized presentation of products and services of a financial institution |
US6360249B1 (en) * | 1998-03-06 | 2002-03-19 | I2 Technologies Us, Inc. | Enterprise interaction hub for managing an enterprise web system |
US20020038217A1 (en) * | 2000-04-07 | 2002-03-28 | Alan Young | System and method for integrated data analysis and management |
US20020099578A1 (en) * | 2001-01-22 | 2002-07-25 | Eicher Daryl E. | Performance-based supply chain management system and method with automatic alert threshold determination |
US6538673B1 (en) * | 1999-08-23 | 2003-03-25 | Divine Technology Ventures | Method for extracting digests, reformatting, and automatic monitoring of structured online documents based on visual programming of document tree navigation and transformation |
-
2001
- 2001-06-08 US US09/877,414 patent/US7188169B2/en not_active Expired - Lifetime
-
2002
- 2002-06-07 WO PCT/US2002/018069 patent/WO2002102097A1/en not_active Application Discontinuation
Patent Citations (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6240295B1 (en) * | 1993-07-20 | 2001-05-29 | @Track Communications, Inc. | Data messaging in a communications network using a feature request |
US6041362A (en) * | 1995-10-20 | 2000-03-21 | Electronics Data Systems Corporation | Method and system for integrating disparate information technology applications and platforms across an enterprise |
US5995943A (en) * | 1996-04-01 | 1999-11-30 | Sabre Inc. | Information aggregation and synthesization system |
US5781703A (en) * | 1996-09-06 | 1998-07-14 | Candle Distributed Solutions, Inc. | Intelligent remote agent for computer performance monitoring |
US6055493A (en) * | 1997-01-29 | 2000-04-25 | Infovista S.A. | Performance measurement and service quality monitoring system and process for an information system |
US6119103A (en) * | 1997-05-27 | 2000-09-12 | Visa International Service Association | Financial risk prediction systems and methods therefor |
US5905868A (en) * | 1997-07-22 | 1999-05-18 | Ncr Corporation | Client/server distribution of performance monitoring data |
US6292830B1 (en) * | 1997-08-08 | 2001-09-18 | Iterations Llc | System for optimizing interaction among agents acting on multiple levels |
US6148335A (en) * | 1997-11-25 | 2000-11-14 | International Business Machines Corporation | Performance/capacity management framework over many servers |
US6108686A (en) * | 1998-03-02 | 2000-08-22 | Williams, Jr.; Henry R. | Agent-based on-line information retrieval and viewing system |
US6360249B1 (en) * | 1998-03-06 | 2002-03-19 | I2 Technologies Us, Inc. | Enterprise interaction hub for managing an enterprise web system |
US6349290B1 (en) * | 1998-06-30 | 2002-02-19 | Citibank, N.A. | Automated system and method for customized and personalized presentation of products and services of a financial institution |
US6538673B1 (en) * | 1999-08-23 | 2003-03-25 | Divine Technology Ventures | Method for extracting digests, reformatting, and automatic monitoring of structured online documents based on visual programming of document tree navigation and transformation |
US20020038217A1 (en) * | 2000-04-07 | 2002-03-28 | Alan Young | System and method for integrated data analysis and management |
US20020099578A1 (en) * | 2001-01-22 | 2002-07-25 | Eicher Daryl E. | Performance-based supply chain management system and method with automatic alert threshold determination |
Cited By (169)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7593893B1 (en) | 2000-06-13 | 2009-09-22 | Fannie Mae | Computerized systems and methods for facilitating the flow of capital through the housing finance industry |
US8244628B1 (en) | 2000-06-13 | 2012-08-14 | Fannie Mae | Computerized systems and methods for facilitating the flow of capital through the housing finance industry |
US7702580B1 (en) | 2000-06-13 | 2010-04-20 | Fannie Mae | System and method for mortgage loan pricing, sale and funding |
US7765151B1 (en) | 2000-06-13 | 2010-07-27 | Fannie Mae | Computerized systems and methods for facilitating the flow of capital through the housing finance industry |
US7139938B2 (en) | 2002-04-01 | 2006-11-21 | Capital One Financial Corporation | System and method for providing common event format using alert index |
US20030200486A1 (en) * | 2002-04-01 | 2003-10-23 | Navjot Marwaha | System and method for providing common event format using alert index |
US20040181685A1 (en) * | 2002-04-01 | 2004-09-16 | Navjot Marwaha | System and method for handling distribution of alerts |
US7437463B1 (en) * | 2002-06-21 | 2008-10-14 | Polycom, Inc. | Method and means for providing scheduling for a videoconferencing network in a manner to ensure bandwidth |
WO2004017243A1 (en) * | 2002-08-15 | 2004-02-26 | Viscom Limited | A data collection, transfer and visualisation network |
US7409461B2 (en) * | 2002-08-19 | 2008-08-05 | Efficient Networks, Inc. | Dynamic file-based routing in a broadband communications system |
US20040034716A1 (en) * | 2002-08-19 | 2004-02-19 | Siemens Westinghouse Power Corporation | Dynamic file-based routing in a broadband communications system |
US9094258B2 (en) | 2002-09-06 | 2015-07-28 | Oracle International Corporation | Method and apparatus for a multiplexed active data window in a near real-time business intelligence system |
US8001185B2 (en) | 2002-09-06 | 2011-08-16 | Oracle International Corporation | Method and apparatus for distributed rule evaluation in a near real-time business intelligence system |
US20080148289A1 (en) * | 2002-09-06 | 2008-06-19 | Oracle International Corporation | Method and apparatus for distributed rule evaluation in a near real-time business intelligence system |
US20080077656A1 (en) * | 2002-09-06 | 2008-03-27 | Oracle International Corporation | Method and apparatus for a report cache in a near real-time business intelligence system |
US8165993B2 (en) * | 2002-09-06 | 2012-04-24 | Oracle International Corporation | Business intelligence system with interface that provides for immediate user action |
US8255454B2 (en) | 2002-09-06 | 2012-08-28 | Oracle International Corporation | Method and apparatus for a multiplexed active data window in a near real-time business intelligence system |
US8566693B2 (en) | 2002-09-06 | 2013-10-22 | Oracle International Corporation | Application-specific personalization for data display |
US8577989B2 (en) | 2002-09-06 | 2013-11-05 | Oracle International Corporation | Method and apparatus for a report cache in a near real-time business intelligence system |
US8402095B2 (en) | 2002-09-16 | 2013-03-19 | Oracle International Corporation | Apparatus and method for instant messaging collaboration |
WO2004059507A1 (en) * | 2002-12-20 | 2004-07-15 | Electronic Data Systems Corporation | Method for monitoring applications in a network which does not natively support monitoring |
US20040122940A1 (en) * | 2002-12-20 | 2004-06-24 | Gibson Edward S. | Method for monitoring applications in a network which does not natively support monitoring |
US20110112955A1 (en) * | 2002-12-30 | 2011-05-12 | Fannie Mae | System and method for pricing loans in the secondary mortgage market |
US20040215554A1 (en) * | 2002-12-30 | 2004-10-28 | Fannie Mae | System and method for verifying loan data at delivery |
US7593889B2 (en) | 2002-12-30 | 2009-09-22 | Fannie Mae | System and method for processing data pertaining to financial assets |
US8671052B1 (en) | 2002-12-30 | 2014-03-11 | Fannie Mae | Method and system for pricing forward commitments for mortgage loans and for buying committed loans |
US8666879B1 (en) | 2002-12-30 | 2014-03-04 | Fannie Mae | Method and system for pricing forward commitments for mortgage loans and for buying committed loans |
US20040128229A1 (en) * | 2002-12-30 | 2004-07-01 | Fannie Mae | System and method for processing data pertaining to financial assets |
US20050102229A1 (en) * | 2002-12-30 | 2005-05-12 | Kemper John L. | Internet-enabled interface for a loan commitment system |
US8515861B2 (en) | 2002-12-30 | 2013-08-20 | Fannie Mae | System and method for facilitating sale of a loan to a secondary market purchaser |
US7742981B2 (en) | 2002-12-30 | 2010-06-22 | Fannie Mae | Mortgage loan commitment system and method |
US8423450B2 (en) | 2002-12-30 | 2013-04-16 | Fannie Mae | System and method for processing data pertaining to financial assets |
US7747519B2 (en) | 2002-12-30 | 2010-06-29 | Fannie Mae | System and method for verifying loan data at delivery |
US20050102226A1 (en) * | 2002-12-30 | 2005-05-12 | Dror Oppenheimer | System and method of accounting for mortgage related transactions |
US20040215555A1 (en) * | 2002-12-30 | 2004-10-28 | Fannie Mae | System and method for creating and tracking agreements for selling loans to a secondary market purchaser |
US20090076973A1 (en) * | 2002-12-30 | 2009-03-19 | Kemper John L | System and method for creating and tracking agreements for selling loans to a secondary market purchaser |
US7809633B2 (en) | 2002-12-30 | 2010-10-05 | Fannie Mae | System and method for pricing loans in the secondary mortgage market |
US20070016520A1 (en) * | 2002-12-30 | 2007-01-18 | Gang John E | System and method for facilitating sale of a loan to a secondary market purchaser |
US7461020B2 (en) | 2002-12-30 | 2008-12-02 | Fannie Mae | System and method for creating and tracking agreements for selling loans to a secondary market purchaser |
US20040225596A1 (en) * | 2002-12-30 | 2004-11-11 | Fannie Mae | System and method for facilitating delivery of a loan to a secondary mortgage market purchaser |
US20040225594A1 (en) * | 2002-12-30 | 2004-11-11 | Fannie Mae | System and method for pricing loans in the secondary mortgage market |
US8065211B2 (en) | 2002-12-30 | 2011-11-22 | Fannie Mae | System and method for creating and tracking agreements for selling loans to a secondary market purchaser |
US8060440B2 (en) | 2002-12-30 | 2011-11-15 | Fannie Mae | System and method for modifying attribute data pertaining to financial assets in a data processing system |
US9928546B2 (en) | 2002-12-30 | 2018-03-27 | Fannie Mae | System and method for processing data pertaining to financial assets |
US8032450B2 (en) | 2002-12-30 | 2011-10-04 | Fannie Mae | Loan commitment system and method |
US8024265B2 (en) | 2002-12-30 | 2011-09-20 | Fannie Mae | System and method for verifying loan data at delivery |
US20040225597A1 (en) * | 2002-12-30 | 2004-11-11 | Fannie Mae | System and method for processing data pertaining to financial assets |
US20080120211A1 (en) * | 2002-12-30 | 2008-05-22 | Dror Oppenheimer | System and method for modifying attribute data pertaining to financial assets in a data processing system |
US20040225584A1 (en) * | 2002-12-30 | 2004-11-11 | Fannie Mae | System and method for defining loan products |
US20040220874A1 (en) * | 2002-12-30 | 2004-11-04 | Fannie Mae | System and method for defining loan products |
US7979346B2 (en) | 2002-12-30 | 2011-07-12 | Fannie Mae | System and method for pricing loans in the secondary mortgage market |
US20040220873A1 (en) * | 2002-12-30 | 2004-11-04 | Fannie Mae | System and method for defining loan products |
US20100268641A1 (en) * | 2002-12-30 | 2010-10-21 | Fannie Mae | System and method for verifying loan data at delivery |
US7885889B2 (en) | 2002-12-30 | 2011-02-08 | Fannie Mae | System and method for processing data pertaining to financial assets |
US7860787B2 (en) | 2002-12-30 | 2010-12-28 | Fannie Mae | System and method for modifying attribute data pertaining to financial assets in a data processing system |
US20100312684A1 (en) * | 2002-12-30 | 2010-12-09 | Kemper John L | Loan commitment system and method |
US20040153329A1 (en) * | 2003-02-03 | 2004-08-05 | Fabio Casati | System and method for monitoring event based systems |
US7437675B2 (en) * | 2003-02-03 | 2008-10-14 | Hewlett-Packard Development Company, L.P. | System and method for monitoring event based systems |
US7603373B2 (en) | 2003-03-04 | 2009-10-13 | Omniture, Inc. | Assigning value to elements contributing to business success |
US20050114510A1 (en) * | 2003-03-04 | 2005-05-26 | Error Brett M. | Assigning value to elements contributing to business success |
US20120239675A1 (en) * | 2003-03-04 | 2012-09-20 | Error Brett M | Associating Website Clicks with Links on a Web Page |
US20040260591A1 (en) * | 2003-06-17 | 2004-12-23 | Oracle International Corporation | Business process change administration |
US10318598B2 (en) | 2003-06-27 | 2019-06-11 | Adobe Inc. | One-click segmentation definition |
US8046298B1 (en) | 2003-07-21 | 2011-10-25 | Fannie Mae | Systems and methods for facilitating the flow of capital through the housing finance industry |
US20050065807A1 (en) * | 2003-09-23 | 2005-03-24 | Deangelis Stephen F. | Systems and methods for optimizing business processes, complying with regulations, and identifying threat and vulnerabilty risks for an enterprise |
US20050065941A1 (en) * | 2003-09-23 | 2005-03-24 | Deangelis Stephen F. | Systems for optimizing business processes, complying with regulations, and identifying threat and vulnerabilty risks for an enterprise |
US7813947B2 (en) * | 2003-09-23 | 2010-10-12 | Enterra Solutions, Llc | Systems and methods for optimizing business processes, complying with regulations, and identifying threat and vulnerabilty risks for an enterprise |
US8423451B1 (en) | 2003-12-01 | 2013-04-16 | Fannie Mai | System and method for processing a loan |
US8489498B1 (en) | 2003-12-01 | 2013-07-16 | Fannie Mae | System and method for processing a loan |
US7925579B1 (en) | 2003-12-01 | 2011-04-12 | Fannie Mae | System and method for processing a loan |
US7653592B1 (en) | 2003-12-01 | 2010-01-26 | Fannie Mae | System and method for processing a loan |
US8417682B2 (en) | 2003-12-12 | 2013-04-09 | International Business Machines Corporation | Visualization of attributes of workflow weblogs |
US20050132048A1 (en) * | 2003-12-12 | 2005-06-16 | International Business Machines Corporation | Role-based views access to a workflow weblog |
US8140691B2 (en) | 2003-12-12 | 2012-03-20 | International Business Machines Corporation | Role-based views access to a workflow weblog |
US20050198021A1 (en) * | 2003-12-12 | 2005-09-08 | International Business Machines Corporation | Visualization of attributes of workflow weblogs |
US20050131750A1 (en) * | 2003-12-12 | 2005-06-16 | International Business Machines Corporation | Method for tracking the status of a workflow using weblogs |
US8423394B2 (en) | 2003-12-12 | 2013-04-16 | International Business Machines Corporation | Method for tracking the status of a workflow using weblogs |
US7822680B1 (en) | 2003-12-31 | 2010-10-26 | Fannie Mae | System and method for managing data pertaining to a plurality of financial assets for multifamily and housing developments |
US7813990B1 (en) | 2003-12-31 | 2010-10-12 | Fannie Mae | Property investment rating system and method |
US7657475B1 (en) | 2003-12-31 | 2010-02-02 | Fannie Mae | Property investment rating system and method |
US7302421B2 (en) | 2004-03-17 | 2007-11-27 | Theoris Software, Llc | System and method for transforming and using content in other systems |
US20050210052A1 (en) * | 2004-03-17 | 2005-09-22 | Aldridge Gregory E | System and method for transforming and using content in other systems |
US20050216831A1 (en) * | 2004-03-29 | 2005-09-29 | Grzegorz Guzik | Key performance indicator system and method |
US7822662B2 (en) * | 2004-03-29 | 2010-10-26 | Microsoft Corporation | Key performance indicator system and method |
US20060059026A1 (en) * | 2004-08-24 | 2006-03-16 | Oracle International Corporation | Compliance workbench |
US20060089861A1 (en) * | 2004-10-22 | 2006-04-27 | Oracle International Corporation | Survey based risk assessment for processes, entities and enterprise |
US20060089889A1 (en) * | 2004-10-22 | 2006-04-27 | 1St Cfo Llc | System for visualizing financial data |
US7788146B2 (en) * | 2004-10-22 | 2010-08-31 | Cfo Technology Corporation | System for visualizing financial data |
US7849396B2 (en) * | 2004-10-29 | 2010-12-07 | International Business Machines Corporation | Method and system for displaying prioritization of metric values |
US20060095282A1 (en) * | 2004-10-29 | 2006-05-04 | International Business Machines Corporation | Method and system for displaying prioritization of metric values |
US20060161471A1 (en) * | 2005-01-19 | 2006-07-20 | Microsoft Corporation | System and method for multi-dimensional average-weighted banding status and scoring |
US20060198208A1 (en) * | 2005-03-07 | 2006-09-07 | Lantronix, Inc. | Publicasting systems and methods |
US7991732B2 (en) | 2005-06-03 | 2011-08-02 | Adobe Systems Incorporated | Incrementally adding segmentation criteria to a data set |
US9081863B2 (en) | 2005-06-03 | 2015-07-14 | Adobe Systems Incorporated | One-click segmentation definition |
US20060277211A1 (en) * | 2005-06-03 | 2006-12-07 | Error Brett M | Incrementally adding segmentation criteria to a data set |
US20060277212A1 (en) * | 2005-06-03 | 2006-12-07 | Error Brett M | User interface providing summary information or a status pane in a web analytics tool |
US9171093B2 (en) | 2005-06-03 | 2015-10-27 | Adobe Systems Incorporated | User interface providing summary information or a status pane in a web analytics tool |
US7941394B2 (en) * | 2005-06-03 | 2011-05-10 | Adobe Systems Incorporated | User interface providing summary information or a status pane in a web analytics tool |
US20060277198A1 (en) * | 2005-06-03 | 2006-12-07 | Error Brett M | One-click segmentation definition |
US7801809B1 (en) | 2005-06-24 | 2010-09-21 | Fannie Mae | System and method for management of delegated real estate project reviews |
US20070094380A1 (en) * | 2005-10-20 | 2007-04-26 | Samuel Fu | Method and apparatus for collocating application monitoring reports with web applications |
US20080301444A1 (en) * | 2005-12-07 | 2008-12-04 | Electronics & Telecommunications Research Institut | Apparatus and Method for Providing Personal Information Sharing Service Using Signed Callback Url Message |
US20070143306A1 (en) * | 2005-12-15 | 2007-06-21 | Jae-Hyuk Yang | Integrated website management system and management method thereof |
US20070143161A1 (en) * | 2005-12-21 | 2007-06-21 | Microsoft Corporation | Application independent rendering of scorecard metrics |
US8712813B2 (en) | 2006-01-05 | 2014-04-29 | Oracle International Corporation | Audit planning |
US20110119107A1 (en) * | 2006-01-05 | 2011-05-19 | Oracle International Corporation | Audit planning |
US20090112663A1 (en) * | 2006-01-30 | 2009-04-30 | Jay William Benayon | Method and apparatus for business process transformation wizard |
US7908161B2 (en) * | 2006-01-30 | 2011-03-15 | International Business Machines Corporation | Method and apparatus for business process transformation wizard |
US8438108B1 (en) | 2006-03-27 | 2013-05-07 | Fannie Mae | System and method for transferring mortgage loan servicing rights |
US7747526B1 (en) | 2006-03-27 | 2010-06-29 | Fannie Mae | System and method for transferring mortgage loan servicing rights |
US20070239573A1 (en) * | 2006-03-30 | 2007-10-11 | Microsoft Corporation | Automated generation of dashboards for scorecard metrics and subordinate reporting |
US20070239660A1 (en) * | 2006-03-30 | 2007-10-11 | Microsoft Corporation | Definition and instantiation of metric based business logic reports |
US7716592B2 (en) * | 2006-03-30 | 2010-05-11 | Microsoft Corporation | Automated generation of dashboards for scorecard metrics and subordinate reporting |
US8261181B2 (en) | 2006-03-30 | 2012-09-04 | Microsoft Corporation | Multidimensional metrics-based annotation |
US7840896B2 (en) | 2006-03-30 | 2010-11-23 | Microsoft Corporation | Definition and instantiation of metric based business logic reports |
US8190992B2 (en) | 2006-04-21 | 2012-05-29 | Microsoft Corporation | Grouping and display of logically defined reports |
US20070265863A1 (en) * | 2006-04-27 | 2007-11-15 | Microsoft Corporation | Multidimensional scorecard header definition |
US7716571B2 (en) | 2006-04-27 | 2010-05-11 | Microsoft Corporation | Multidimensional scorecard header definition |
US20070282824A1 (en) * | 2006-05-31 | 2007-12-06 | Ellingsworth Martin E | Method and system for classifying documents |
US8255347B2 (en) | 2006-05-31 | 2012-08-28 | Hartford Fire Insurance Company | Method and system for classifying documents |
US20110047168A1 (en) * | 2006-05-31 | 2011-02-24 | Ellingsworth Martin E | Method and system for classifying documents |
US8738552B2 (en) | 2006-05-31 | 2014-05-27 | Hartford Fire Insurance Company | Method and system for classifying documents |
US7849030B2 (en) | 2006-05-31 | 2010-12-07 | Hartford Fire Insurance Company | Method and system for classifying documents |
US10453029B2 (en) | 2006-08-03 | 2019-10-22 | Oracle International Corporation | Business process for ultra transactions |
US20080183519A1 (en) * | 2006-08-03 | 2008-07-31 | Oracle International Corporation | Business process for ultra vires transactions |
US20080162204A1 (en) * | 2006-12-28 | 2008-07-03 | Kaiser John J | Tracking and management of logistical processes |
US20080172629A1 (en) * | 2007-01-17 | 2008-07-17 | Microsoft Corporation | Geometric Performance Metric Data Rendering |
US20080172414A1 (en) * | 2007-01-17 | 2008-07-17 | Microsoft Corporation | Business Objects as a Service |
US20080178148A1 (en) * | 2007-01-19 | 2008-07-24 | International Business Machines Corporation | Business performance bookmarks |
US10515329B2 (en) | 2007-01-19 | 2019-12-24 | International Business Machines Corporation | Business performance bookmarks |
US11195136B2 (en) | 2007-01-19 | 2021-12-07 | International Business Machines Corporation | Business performance bookmarks |
US20080184099A1 (en) * | 2007-01-26 | 2008-07-31 | Microsoft Corporation | Data-Driven Presentation Generation |
US9058307B2 (en) * | 2007-01-26 | 2015-06-16 | Microsoft Technology Licensing, Llc | Presentation generation using scorecard elements |
US20080184130A1 (en) * | 2007-01-30 | 2008-07-31 | Microsoft Corporation | Service Architecture Based Metric Views |
US20080183564A1 (en) * | 2007-01-30 | 2008-07-31 | Microsoft Corporation | Untethered Interaction With Aggregated Metrics |
US8321805B2 (en) | 2007-01-30 | 2012-11-27 | Microsoft Corporation | Service architecture based metric views |
US20080189632A1 (en) * | 2007-02-02 | 2008-08-07 | Microsoft Corporation | Severity Assessment For Performance Metrics Using Quantitative Model |
US9392026B2 (en) | 2007-02-02 | 2016-07-12 | Microsoft Technology Licensing, Llc | Real time collaboration using embedded data visualizations |
US8495663B2 (en) | 2007-02-02 | 2013-07-23 | Microsoft Corporation | Real time collaboration using embedded data visualizations |
US8214846B1 (en) * | 2007-09-12 | 2012-07-03 | Sprint Communications Company L. P. | Method and system for threshold management |
US20090132304A1 (en) * | 2007-11-16 | 2009-05-21 | Sap Ag | Determining a Value for an Indicator |
US8255245B2 (en) * | 2007-11-16 | 2012-08-28 | Sap Ag | Determining a value for an indicator |
US20090183104A1 (en) * | 2008-01-03 | 2009-07-16 | Dotson Gerald A | Multi-mode viewer control for viewing and managing groups of statistics |
US8880459B2 (en) | 2008-03-07 | 2014-11-04 | Microsoft Corporation | Navigation across datasets from multiple data sources based on a common reference dimension |
US20090228485A1 (en) * | 2008-03-07 | 2009-09-10 | Microsoft Corporation | Navigation across datasets from multiple data sources based on a common reference dimension |
US20100073160A1 (en) * | 2008-09-25 | 2010-03-25 | Microsoft Corporation | Alerting users using a multiple state status icon |
US20100082702A1 (en) * | 2008-09-29 | 2010-04-01 | Honeywell International Inc. | Dynamic vehicle information management |
US20130013370A1 (en) * | 2008-12-30 | 2013-01-10 | Infosys Limited | System and method for automatically generating an optimized business process design |
US20110004627A1 (en) * | 2009-07-01 | 2011-01-06 | Oracle International Corporation | Dashboard for business process management system |
US20120191831A1 (en) * | 2011-01-26 | 2012-07-26 | Carl Kanzabedian | System and method for cataloging assets in a network |
US9722891B2 (en) * | 2011-01-27 | 2017-08-01 | 10029629 Canada Inc. | Method and system for generating and providing data alerts |
US20130304910A1 (en) * | 2011-01-27 | 2013-11-14 | Gmarkets Inc. | Method and system for generating and providing data alerts |
US9398475B2 (en) * | 2011-12-26 | 2016-07-19 | Huawei Technologies Co., Ltd. | Method, device, and system for monitoring quality of internet access service of mobile terminal |
US20140334326A1 (en) * | 2011-12-26 | 2014-11-13 | Huawei Technologies Co., Ltd. | Method, Device, and System for Monitoring Quality of Internet Access Service of Mobile Terminal |
US11295247B2 (en) | 2012-03-19 | 2022-04-05 | International Business Machines Corporation | Discovery and generation of organizational key performance indicators utilizing glossary repositories |
US10546252B2 (en) * | 2012-03-19 | 2020-01-28 | International Business Machines Corporation | Discovery and generation of organizational key performance indicators utilizing glossary repositories |
US20130246129A1 (en) * | 2012-03-19 | 2013-09-19 | International Business Machines Corporation | Discovery and realization of business measurement concepts |
US9092282B1 (en) | 2012-08-14 | 2015-07-28 | Sprint Communications Company L.P. | Channel optimization in a messaging-middleware environment |
US8793236B2 (en) | 2012-11-01 | 2014-07-29 | Adobe Systems Incorporated | Method and apparatus using historical influence for success attribution in network site activity |
US9264338B1 (en) | 2013-04-08 | 2016-02-16 | Sprint Communications Company L.P. | Detecting upset conditions in application instances |
US9443214B2 (en) * | 2013-08-30 | 2016-09-13 | Sap Se | News mining for enterprise resource planning |
US20150066567A1 (en) * | 2013-08-30 | 2015-03-05 | Sap Ag | News mining for enterprise resource planning |
US10521601B2 (en) | 2014-04-30 | 2019-12-31 | Sailpoint Technologies, Israel Ltd. | System and method for data governance |
US10402296B2 (en) * | 2014-08-27 | 2019-09-03 | Kabushiki Kaisha Toshiba | Monitor control system and data collection apparatus |
US20230153217A1 (en) * | 2017-03-20 | 2023-05-18 | Netapp Inc. | Methods and systems for managing networked storage system resources |
US11475380B2 (en) * | 2018-05-14 | 2022-10-18 | Horiba, Ltd. | Vehicle test facility operation rate analysis system and method |
US11461677B2 (en) | 2020-03-10 | 2022-10-04 | Sailpoint Technologies, Inc. | Systems and methods for data correlation and artifact matching in identity management artificial intelligence systems |
US11308186B1 (en) | 2021-03-19 | 2022-04-19 | Sailpoint Technologies, Inc. | Systems and methods for data correlation and artifact matching in identity management artificial intelligence systems |
US12298873B2 (en) * | 2022-11-14 | 2025-05-13 | Netapp, Inc. | Methods and systems for managing networked storage system resources |
Also Published As
Publication number | Publication date |
---|---|
WO2002102097A1 (en) | 2002-12-19 |
US7188169B2 (en) | 2007-03-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7188169B2 (en) | System and method for monitoring key performance indicators in a business | |
US11914501B1 (en) | User interface for specifying data stream processing language programs for analyzing instrumented software | |
US10346142B1 (en) | Automated streaming data model generation | |
US7739121B2 (en) | Method and apparatus for providing intelligent and controlled access to supply chain information | |
US8719071B2 (en) | System, method and computer program product for processing and visualization of information | |
US7069233B2 (en) | System and method for identification and notification of elevated over-the-counter medication sales with response coordination | |
WO2018228285A1 (en) | Data acquisition and query methods and apparatuses, storage medium, and processor | |
US20080162210A1 (en) | Configurable goals in a dashborad application | |
US20060089939A1 (en) | Business intelligence system with interface that provides for immediate user action | |
US20110099559A1 (en) | Monitoring Information Assets and Information Asset Topologies | |
US20060200773A1 (en) | Apparatus method and article of manufacture for visualizing status in a compute environment | |
US20120151396A1 (en) | Rendering an optimized metrics topology on a monitoring tool | |
US20120254053A1 (en) | On Demand Information Network | |
US10007951B2 (en) | IT asset management trend charting for compliance over time | |
US20040138931A1 (en) | Trend detection in an event management system | |
US20130346163A1 (en) | Automatically measuring the quality of product modules | |
US6721750B1 (en) | System and method for broadband capacity tracking | |
US20070162485A1 (en) | Generating contextual support requests | |
US7657451B2 (en) | Six sigma enabled web-based business intelligence system | |
US9448998B1 (en) | Systems and methods for monitoring multiple heterogeneous software applications | |
US7680823B2 (en) | Custom report generation | |
US20070162494A1 (en) | Embedded business process monitoring | |
US9769249B2 (en) | Impact analysis of service modifications in a service oriented architecture | |
US20230289035A1 (en) | System and Method for Visual Data Reporting | |
Jamalzadeh | Analysis of clickstream data |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: XOR, INC., COLORADO Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BUUS, BRYAN;GAZ, RANDALL;MALAHER, TOM;AND OTHERS;REEL/FRAME:012237/0091;SIGNING DATES FROM 20010629 TO 20010802 |
|
AS | Assignment |
Owner name: SEURAT, INC., MASSACHUSETTS Free format text: SECURITY INTEREST;ASSIGNOR:XOR, INC.;REEL/FRAME:012787/0929 Effective date: 20020319 |
|
AS | Assignment |
Owner name: SILLICON VALLEY BANK, CALIFORNIA Free format text: SECURITY AGREEMENT;ASSIGNOR:XOR, INC;REEL/FRAME:013009/0388 Effective date: 20020524 |
|
AS | Assignment |
Owner name: FAIR ISAAC CORPORATION, MINNESOTA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SEURAT COMPANY;XOR, INC.;RAPID INSIGHTS, INC.;REEL/FRAME:014030/0325;SIGNING DATES FROM 20030920 TO 20030930 |
|
AS | Assignment |
Owner name: RAPID INSIGHTS, INC., COLORADO Free format text: TERMINATION OF SECURITY INTEREST;ASSIGNOR:SILICON VALLEY BANK;REEL/FRAME:014044/0930 Effective date: 20031001 Owner name: XOR, INC., COLORADO Free format text: TERMINATION OF SECURITY INTEREST;ASSIGNOR:SILICON VALLEY BANK;REEL/FRAME:014044/0930 Effective date: 20031001 Owner name: SEURAT COMPANY, COLORADO Free format text: TERMINATION OF SECURITY INTEREST;ASSIGNOR:SILICON VALLEY BANK;REEL/FRAME:014044/0930 Effective date: 20031001 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
CC | Certificate of correction | ||
FPAY | Fee payment |
Year of fee payment: 4 |
|
FPAY | Fee payment |
Year of fee payment: 8 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 12 |