SYSTEM AND METHOD TO DETECT PHISHING AND VERIFY ELECTRONIC ADVERTISING
TECHNICAL FIELD
This invention relates generally to information delivery and management in a computer network. More particularly, the invention relates to techniques for automatically classifying electronic communications and web pages as phishing or non-phishing. It also relates to techniques for detecting viruses or worms.
This application claims the benefit of the filing date of U.S. Provisional Application, Number 60/522245, "System and Method to Detect Phishing and Verify Electronic Advertising", filed September 7, 2004, which is incorporated by reference in its entirety. It also incorporates by reference in their entireties the U.S. Provisional Application, Number 60/522458, "System and Method for Enhanced Detection of Phishing", filed October 4, 2004, and U.S. Provisional Application, Number 60/552528, "System and Method for Finding Message Bodies in Web-Displayed Messaging", filed October 11, 2004.
BACKGROUND OF THE INVENTION
Phishing is characterized by the phisher sending many electronic messages, typically purporting to be from a financial institution or large company, at which the recipient presumably has an account. Often, the message will claim that the recipient needs to enter certain crucial personal pieces of information, like her account number or username, and her password. The message usually looks like it came from the purported source. It may have images downloaded from the actual source, to reinforce this impression.
But it may have, and this is the point of the message, a link or button that is actually to a
different location. So if the user fills out her personal data in the message and presses the button, the information is sent to the phisher. Or if she follows the link, she gets to a page at some website where she is encouraged to fill out information, and this page is then sent to the phisher. Alternatively, but with the same end effect, the message might have a button that makes a popup window. Within this popup might be a form to be filled out and a button to be pushed that sends the data to the phisher.
Phishing is a type of spam. And it is well known that many spam messages have forged senders. But phishing tends to be fairly distinct from other spam, in that its forged sender is meant to have specific deceptive purpose. Usually reinforced by wording and visuals in the body of the message.
Phishing messages tend to be very carefully crafted. Unlike a typical spam message, phishing is considered highly damaging, because it involves actual fraud. There is incentive for an ISP for prevent it, as well as for the company that is impersonated by the phisher.
Also, some phishers have released viruses that take over computers, often unbeknownst to their owners. These viruses then run web servers that are the destinations of those crucial links in mass phishing messages. The viruses forward any such information received from tricked users to other locations on the Internet. In this way, the phisher can further obscure her origin. Plus, for a given mass phishing mailing, she can divide up those messages so that different messages might point to different hijacked computers. This means that tracking down and disabling the virus on one such computer is not enough to stop all her phishing.
Recently targeted companies in 2004 included Citicorp, eBay and its PayPal subsidiary,
US Bank, Lloyds, Barclays, HSBC and Westpac. There were 1422 unique phishing attacks reported in June 2004. Of these, at least 92% used forged sender addresses. Some 2-3% used "social engineering", where the sender domain was a real domain with a name
close to that of the target. For example, for the Visa credit card organization, one sender address used was support@verify-visa.org. The domain in this instance had no affiliation with Visa. ["Phishing Attack Trends Report", June 2004, by the Anti-Phishing Working Group.]
Avivah Litan, research director of Gartner Inc., estimated that as of April 2004, phishing victims lost $1.2 billion.
Plus, until 2004, most known phishing was against companies in the US, UK and Australia. It is spreading to more countries. In June 2004, two major German banks,
Deutsche Bank and Postbank reported the first attacks against them. There are no technological constraints against phishing spreading to any country. To this ends, for example, the strong growth of the Chinese and Indian economies, and the massive buildup of their electronic networks means that their banks will eventually be hit by this plague.
Various methods have been tried against phishing. One method compiles blacklists of phishing domains. These have limited efficacy. The domains are short lived, since the ISPs or NSPs hosting them often quickly take them down, when informed about the phishing activity. Hence, such a blacklist is largely moot, since by the time a domain gets into this list, it is no longer operating. Plus, the method affords no protection in the time between the domain coming online and phishing messages sent out, and when some organization discovers the phishing and puts the domain into its blacklist, and informs the phishing domain's ISP or NSP.
Another method looks for heuristics in a suspected message. Like key words or phrases. This has the drawback of language dependence. Plus, it is probabilistic (subjective). It does not decisively (deterministically) say whether a message is phishing or not. This method comes from its use against general spam.
Another method looks at registrations of new domains. It flags a domain if that name seems to be a variant of a bank's name, say. But a phisher might register a completely innocuous domain. And then possibly use a bank's name as a subdomain. Where she does not have to register this subdomain with any DNS group. Hence it is not possible to detect this subdomain until after she sends out her phishing messages. Plus, the method fails against her subverting an existing domain and using that as a destination in her messages.
Another method involves spiders crawling the Web. Looking for a phisher setting up a domain with web pages that mimic a bank's. But the Web is so vast that where does this method start looking? Also, a smart phisher can isolate her domain from the Web while she is developing it. And, if she needs to test it connected to the Web, she can just make it connected for a brief time.
Another method involves an ISP using Bayesians to try to detect phishing. Bayesians are popular for antispam. (Though they have limitations there.) However, there is a fundamental problem with using Bayesians against phishing. A Bayesian needs two sets of training data. One set is good data. The other is bad. In this latter case, phishing messages. But these use very respectable and mainstream financial language. They adhere closely to legitimate financial messages. Wheres generic spam often has sexual words, or phrases about "free" or health supplements [etc].
Another method involves user education. But there will always be gullible users. And the phishers can produce messages of extremely high quality, with very convincing text and visuals. Along these lines, one variant method tells the user to move her mouse over the links in a suspected message. Looking for a link that does not go to the purported sender. There are several problems with this. Firstly, it is manually some effort to move the mouse over several links and look at what the browser says is the destination. There are unsophisticated users who do not understand what the browser is saying for the destination. They know enough to identify and click on a link. But the destination
information is beyond their ken. Secondly, that information can be deliberately obfuscated by the phisher. She might write a long address, with the leftmost part containing what appears to be the bank's domain. Or she might have a domain with a name like the bank's. Users can get fooled here too. Or the browser might have a bug that lets her write a widget over portions of the browser, to obscure information presented by the browser. Thirdly, this method involves steps done on suspected messages. Because no one can be expected to do this for all messages. But the method will fail if the user gets a message that she does not suspect.
A variation of the user education involves suggesting that the user examine the HTML source code of a suspected phishing message. So that she might find the actual network address that a link goes to, and see if this is suspect. It is difficult to take this seriously. Most HTML email that is read in a browser has a structure that someone not used to HTML will find hard to follow. In part because the message providers that offer a web interface to read email often add many ads into the web page, outside the region displaying the message. Plus, for the user to examine the HTML, she has to decide in the first place that the message appears suspicious.
Another method uses two factor gadgets. (A hardware cryptographic approach.) These generate transient passwords that the user can use when logging into a bank's website. So that if the website is fake ("pharm"), the phisher's capture of the password is of little value. But the gadgets were developed for use by employees and consultants of a company, when remotely accessing the company's computers. Usually, an employee only has one full time job, and so only needs to carry one gadget. But people often have several bank accounts, plus perhaps accounts at other companies targeted by phishers. It is very cumbersome for a person to carry around several gadgets.
Another method involves a software cryptographic approach. Where a bank might distribute passwords to its customers. And thence possibly encrypt future messages to them. An unsophisticated customer might find this confusing. And there are problems
with key distribution and revocation.
In light of the above, it would be desirable to have a method that is objective, easy to use, especially by a mass audience, and computationally lightweight.
SUMMARY OF THE INVENTION
The foregoing has outlined some of the more pertinent objects and features of the present invention. These objects and features should be construed to be merely illustrative of some of the more prominent features and applications of the invention. Other beneficial results can be achieved by using the disclosed invention in a different manner or changing the invention as will be described. Thus, other objects and a fuller understanding of the invention may be had by referring to the following detailed description of the Preferred Embodiment.
We attack the problem of phishing in electronic messages. Our method involves a company producing, optionally but preferably as a Web Service, a list of partner domains or addresses that will be in links in outgoing messages or in its web pages. ISPs and other message providers can access these lists and apply them against domains inside bodies of incoming messages purporting to be from that company. If a domain in such a message is not on the Partner List, then the message can be dealt with as a phishing message or invalid electronic advertisement. An ISP can also use the method to permit advertising access to its customers. The method defines a heterogeneous p2p network whose main purpose is to prevent phishing fraud. The method also involves a third party (Aggregator) that collects Partner Lists and acts as a trusted source of these for network queries.
On a browser, we implement a plug-in that can be used to validate links and text in a web page and in messages viewed in the browser. This is used in conjunction with a custom tag that we introduce, and the Aggregator that is queried for the Partner List of an address
in the tag. A lightweight alternative to other methods that attempt to authenticate messages or web pages. We call the custom tag a Notphish tag.
We show how a Notphish tag inside data released by a company can be used to tackle the problem of viruses or other malware in distributed data.
Our methods offer a unified detection of phishing, pharming and viruses.
BRIEF DESCRIPTION OF THE DRAWINGS
Fig. 1 shows a systems diagram of the general Anti -phishing technology.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
What we claim as new and desire to secure by letters patent is set forth in the following claims.
Fig. 1 shows a systems diagram of the general Anti-phishing technology.
We offer a lightweight and objective method for attacking phishing. It involves actions performed by several parties: The first party is the companies that have been impersonated or think that they might be impersonated in the future by phishers. The second party is the ISP or other electronic message providers. Later, we show how other parties can be involved.
Our methods apply across all electronic communication modalities, including email, instant messaging, WAP (Wireless Access Protocol) and SMS.
For brevity in what follows, we refer to a recipient of a message viewing or reading that message. Our method also applies when the message is heard by the user. For example, the user might be blind, and the "browser" is a program that reads out the message to her. Also for brevity, we imagine that the first party might be a bank.
Our method is divided into five sections - Partner List, Notphish Tag, Finding Message Bodies, Antivirus Method, Summary.
Section 1. Partner List
1.1 A Bank
Consider a company, for example BankO, with a website, bankO.com. On its website's IP address, at some port, it offers a programmatic interface. An external party, like an ISP, can query this. Call this party ISPO. BankO replies with a string that is a list of base domains in links in messages that it will be sending or is sending to its customers or anyone else, like potential customers. The query and the reply can be in any format that BankO and a potential questioner, who might be a message provider, agree to support.
Optionally, this format could be in XML. For example, the query could simply be "<askLinks/>". The reply by BankO could be
<Links>
<p>bank0.com</p> <p>partner0.com</p> <p>partner 1.com</p> <p>partner2. com. au</p> <p>partner3.fr</p>
</Links>
Here, when we say "base domains", we mean that you take a domain, like a.b.partnerO.com, and reduce it to the minimal rightmost name that can be owned. Which gives us partnerO.com in this example. Notice that this means that the number of fields in the resultant base domain can be a function of the country, if the rightmost field is a country. This is also shown above, where we have the base domain partner2.com.au, because in Australia (represented by "au"), its dot com domains end in com.au. Whereas in France (represented by "fr"), you can own a domain, like "partner3.fr".
Optionally, a raw network address might be used, instead of a base domain.
Optionally, the syntax of the query and reply might be more intricate. For example, in the reply, an entry for a partner might include the start and end dates for which messages were sent that had links to that partner. This would let ISPO assess a message for validity at the time when it was received. Another example might be a notation for a raw network address that gave a range of addresses. For instance, using the Internet Protocol Version 4, a reply might include "10.20.30.*" where the asterisk denotes any valid value in the last field.
As another example, the bank might also return information about the various stylistic properties of its outgoing messages. We defined various heuristic properties of messages, which we termed "styles" in previous U.S. Provisional filings: Number 60320046, "System and Method for the Classification of Electronic Communications", filed March 24, 2003; Number 60481745, "System and Method for the Algorithmic Categorization and Grouping of Electronic Communications, filed December 5, 2003; Number 60481789, "System and Method for the Algorithmic Disposition of Electronic Communications", filed December 14, 2003; Number 60481899, "Systems and Method for Advanced Statistical Categorization of Electronic Communications", filed January 15, 2004; Number 60521014, "Systems and Method for the Correlations of Electronic Communications", filed February 5, 2004; Number 60521174, "System and Method for
Finding and Using Styles in Electronic Communications", filed March 3, 2004; Number 60521622, "System and Method for Using a Domain Cloaking to Correlate the Various Domains Related to Electronic Messages", filed June 7, 2004; Number 60521698, "System and Method Relating to Dynamically Constructed Addresses in Electronic Messages", filed June 20, 2004; Number 60521942, "System and Method to Categorize Electronic Messages by Graphical Analysis", filed July 23, 2004; Number 60522244, "System and Method to Rank Electronic Messages", filed September 7, 2004; Number 60522113, "System and Method to Detect Spammer Probe Accounts", filed August 17, 2004.
In the following text, we shall refer to these collectively as "Earlier" Provisionals.
The bank might make these styles available to other organizations. So an ISP which gets a message can reduce it to a canonical form, as in Provisional 60320046, and then find its • styles and compare these with the approved styles for that bank. If some styles of the message are not in the approved styles, then the ISP might regard the message as not being from the bank. Along these lines, the bank might also return full or partial Bulk Message Envelope [BME] information, where the BME is defined in that Provisional. (The styles are partial BME information.)
Of course, in these cases, this extra information is not a "Partner List", per se. But for simplicity, we continue to use that term to describe all the information sent by BankO to ISPO.
Another example might be where the query has a domain, and is asking whether that domain is on the Partner List. For example, the query might be "<aslcLinks>partner20.com</askLinks>". Then the reply might simply be "<yes/>" or "<no/>". BankO might want to do this if it wants to hide its Partner List. So that someone who wants to know has to make repeated queries. This can be detected by BankO and used as a possible heuristic indicating a non-message provider, if the questioner offers a
set of domains that tend to not be in the Partner List. That is, the questioner seems to be guessing, as opposed to finding these in actual messages.
We recommend that in most cases, these advanced features not be used unless there is a pressing need for them.
If the query and reply are in XML, then we have an instance of a simple Web Service. Currently, most Web Services are highly complex, due to richly structured data that is under consideration. In contrast, our method is easy to implement for the company.
We also include the general case where the company transmits this information to the ISP by any means, including non-electronic. For example, suppose Laura is BankO's person that handles this matter. And Dinesh is her counterpart at ISPO. Laura and Dinesh meet in person. She hands over a piece of paper with the list of partner domains. Dinesh then takes this back to ISPO and manually types this into ISPO's computer's Partner List for BankO.
Of course, for whatever method is chosen to transmit the information, care should be taken by both parties to reduce the risk of a phisher introducing false information. For example, suppose Laura and Dinesh decide that Laura will send the information by email. Suppose her address is laura@bankO.com. Clearly, when Dinesh gets a message purporting to be from that address, he should have some other assurance that it is authentic. Like talking to her, to confirm this. Or the message might be signed with her private key, and Dinesh can then use her public key to authenticate.
In the above, when we said "partner", this can also include subsidiaries and parent companies. For example, suppose BankO has a home mortgage subsidiary, HomeO. Perhaps it bought this company at some time in the past, and runs it as a wholly-owned subsidiary that has its own website. Then the Partner List can include homeO.com. Likewise, suppose BankO is a subsidiary of an automaker, AutoO. It might be the
financing arm for AutoO. Then BankO's Partner List might also have an entry for autoO.com.
1.2 An ISP
Given the above, suppose ISPO has received the Partner List from BankO by whatever means, and entered it into its database. ISPO then receives a message for one of its users, with the sender supposedly being from BankO. That is, the sender purports to be, for example, someone@* .bankO.com, where the asterisk indicates possible subdomains of bankO.com. It is well known that spammers, in general, forge the sender lines. But phishers, in particular, forge these, in most known cases of phishing, because it is crucial that the recipient believe the message is actually from a purported source, starting with the sender line.
ISPO then extracts the base domains from links in the body of the message. Taking care to avoid links that are in comments. During this extraction, if it encounters dynamically generated hyperlinks, it should apply our method in Provisional 60521698 in order to properly extract any valid base domain.
It then compares these base domains with those given by BankO. If ANY are not in BankO's official Partner List, then it can do several actions, including, but not limited to the following:
• Block the message entirely from reaching the recipient. That is, do not even put this into the user's Bulk (or Spam) folder, if the latter exists.
• Send the message to the user's Bulk folder, if the latter exists. Possibly with a warning inserted into the message's header or body.
• Send the message to BankO, with a note warning of a possible phishing or spam message.
• Manually/algorithmically subject the message to more tests.
• Inform the authorities.
As a practical matter, ISPO can maintain a list of companies that are likely to be impersonated. Periodically, it could query those companies for their lists of partners, and then store and use these lists in the above manner. Or there might be some central organization with a list of companies that offer such data about their partners. ISPO can get its list of companies from this organization, and thence query those companies, or get that data from the organization. In the next section, we call this an Aggregator, and will describe its roles.
Optionally, ISPO can reduce the message to a canonical form and put it into a BME as described in Provisional 60320046. Then it can compare information in this BME with any full or partial BME information obtained from BankO. If there is disagreement, the message might be considered as not coming from BankO.
1.3 Remarks
Our method constitutes a simple peer-to-peer [p2p] network. A heterogeneous network, because there are two types of peers - ISPs and the message originators. Currently, many p2p networks have been the subject of much controversy, since a common if not predominant activity on such networks involves copyright infringement. In our p2p network, this is not an issue. Each company generates its own Partner List, and owns that, by copyright. And, by making it available to an ISP, the company explicitly consents to it being used in our method by the ISP. Plus, of course, the main purpose, if not the only purpose, of this p2p network is to prevent fraud.
We also include the case where BankO does not make its Partner List available to a query by any other party. Instead, BankO might require an explicit prior agreement with specific ISPs. Then, it might furnish those ISPs with usernames and passwords. So that an ISP
would give its usemame and password, and, if BankO says these are valid, it replies with its Partner List. Here, optionally, the usemame of an ISP might be one or more of its network addresses.
5 Any company, like BankO, that uses our method can also make its list easily viewable via its home page on the Web. This lets people also manually check their messages against this latter list. Since the list concerns partners that BankO will be, or has been, sending out many messages about, then business confidentiality should not be an issue, about exposing the contents of the list.
10
BankO can still send messages that describe companies not on its Partner List. It just should not make these hyperlinks. Intra-company, users inside BankO can still send any messages to other internal users, and have hyperlinks to any outside destination, independent of the Partner List. BankO can run our method as an ISP, on its messages that
15 come in from outside. It can exempt mail originating internally from our method's tests. Obviously, if it gets a message from outside that claims to be from bankO.com, then it should reject that message anyway, quite aside from using our method.
To BankO, the main thing it has to do is to have a policy decision about implementing this, 20. and thence coordinate between its various departments. Most probably, between the Information Technology and the Sales and Marketing departments. So that IT has, at any given time, a valid Partner List. Contrast this to the effort involved in wire transfers, where there are many precise and highly detailed steps that must be adhered to.
25 It should be noted that many large companies, especially in the financial sector, already carefully vet outgoing messages. Perhaps for regulatory reasons, or just corporate policy. These messages might be reviewed by some internal department for approval, before they are sent out. So the information that our method requires BankO to make available in a Partner List may well already exist, albeit not in this form.
30
When ISPO needs to extract base domains from message bodies, it may already be doing so. For example, in order to apply a blacklist against the messages. If so, then the extra effort in applying our method is minimal. Note, however, that a Partner List is fundamentally different from a blacklist. A user on an ISP might have one or more of a white list, gray list or blacklist. Likewise, the ISP might also maintain default white, gray or blacklists. Typically used against incoming messages, where the user is now a recipient. Our Partner Lists are associated with authentic senders, not users or the ISP itself.
But there is a deeper reason why our Partner Lists are different from a blacklist. An entry in a blacklist is an undesired network address. Typically, if you find a blacklist entry in the sender field or in a link in the body of a message under scrutiny, then you reject the message or mark it as Bulk. Because of this, spammers, including phishers, have incentive to continually set up new domains. This might be via registering new domains and then running web servers for those domains. Or by taking over computers, as described earlier. The effect is that blacklists need to be continuously updated. Plus, at any given time, a blacklist for an ISP can have hundreds of thousands (millions?) of entries. This is because, in the case of the Internet, the name space (i.e. the space of possible domain names) is essentially infinite. The IP address space has around 2Λ32 entries, for IPv4, which is the current common version of the Internet. Plus, when IPv6 becomes common, its numbering space will have 2A128 entries. So spammers have a lot of name and address space to occupy.
Our Partner Lists flip this around. For a given message from a purported company, we look at the message and the company's Partner List. [Assume the latter exists.] All we are looking for is the presence of a link domain not on the Partner List. If so, then we treat the message as bogus, and presumably phishing, without caring as to the value of that outside domain. Although we can, and should, as discussed earlier, take measures against this domain, like alerting others. This completely sidesteps such techniques as the phisher using many hijacked computers as destinations in links. She can use one or a thousand
such machines, and it does not matter.
In this situation, what if the phisher hijacked a computer inside one of the Partners, and used it as a destination in her messages? Yes, this method is vulnerable. But a moment's consideration shows that this scenario actually highlights a strength of the method. Imagine that there is no Partner List for BankO. Then we have the current phishing situation, where a message can pretend to be from BankO, with links to it, and with a phishing link to anywhere else on the net. And if a computer can be hijacked by a virus, say, then typically there will be many of these computers vulnerable and available to the phisher. BankO cannot stop this. Now suppose it has a Partner List. This reduces the number of vulnerable computers by orders of magnitude. From some number that might be a sizable fraction of all computers on the net, to only those at its partners. Plus, BankO should choose its partners carefully. So that they have IT departments maintaining a close watch on their machines, and installing patches regularly.
Another issue is the relative sizes of the tables. A blacklist can have hundreds of thousands or millions of entries. Whereas the number of companies using our method to furnish Partner Lists is likely to be few. Only the largest companies might need our method, because they are the likeliest targets for phishers. Suppose these are the 500 largest companies in a country, and the average size of a Partner List is 10. If we use hashing to speed searching, then we have two levels of searching. The first level is a hash search of a table of 500 entries, so the search time goes as log 500. Given a particular message, and thus a Partner List, assuming it exists, then we have repeated searches of a hash table of size 10, for each link we find in the message, where we stop if we find a link domain not in the Partner List or when we get to the end of the message. So given a message, the search time is roughly proportional to log 10. Let n be the average number of links in a message. Then the total search time goes roughly as log 500 + n * log 10. Now let the blacklist have m entries, m > 10Λ5. Then applying a blacklist to a message goes as n * log m. So applying the blacklist may take longer, if log 500 < n * log (m/10). Obviously, here in general, the 500 and 10 would be replaced by variables. But so long as
500 and 10 are plausible ballpark values of those variables, and n and m take on plausible values of their own, then it should be quicker to apply the Partner Lists.
There is another difference between the Partner Lists and a blacklist. There are many ways to compute a blacklist, and not all of these can be done in anything approaching real time. A spammer, especially a phisher, can set up beforehand a set of new domains and addresses, that have never been used for spamming, and hence might not be on any ISP's blacklist. This is especially true if the domains consist of hijacked computers. Then, she sends a barrage of many messages. The messages may or may not emanate from those domains. But the key characteristic is that the messages have links to these domains [or addresses]. She is taking advantage of the vulnerability of a blacklist to a "zero-day" or "zero-hour" attack. This refers to the fact that it takes time for someone at an ISP or antispam company that maintains a blacklist to identify and put a new entry into the blacklist. During this time, anyone receiving such a message is vulnerable to being tricked.
In response, an ISP might perhaps delay the delivering of received messages to its users' mailboxes, until it has had time to try to compute new entries for the blacklist, based on the new mail. Many ISPs will not be willing or able to do this, especially because it introduces a delay in mail getting for the users. For competitive reasons, this is not attractive. Most users expect to get messages as quickly as possible. In contrast, when applying the Partner Lists, these can be done immediately.
An ISP can decide whether or not to charge for using our method. It might charge BankO. If so, it might also decide not to charge other companies. It can do this on a case by case basis. Regardless of whether it charges any companies or not, it can have separate policies regarding charging its users. For example, it might not charge them at all. Or, it might offer this as part of an extra, for-fee, premium service, as protection against phishing. Also, different ISPs can pursue different policies.
In the above example of a query and reply, elaborations are possible. For example, an ISP, or any other entity on the net, for that matter, can ask BankO to be put on an "alert list". So that when BankO changes its list of partners, it informs everyone on the alert list. This could be done via email. Or perhaps by a Web Services-type interaction. In any case, when BankO informs others of changes, it might include the new list in its message, or it might require that the contactees use the above query and reply method to get the new list.
This is equivalent to saying that the transmission of a Partner List from BankO to ISPO can be initiated by either party.
In the above, we have shown how an ISP can easily apply our method. But it is not restricted to an ISP or message provider. Imagine a browser or any other program that can display an electronic message that contains hyperlinks, or buttons linking to somewhere on an electronic network, and can let the user pick such a link or button and thence go to that link, or destination in the button. This program can use our method. It can have a list of companies, for which it periodically goes to, in the fashion described for ISPs earlier, and downloads their Partner Lists. Then, when the user views a message that claims to be from a sender at one of those companies, it checks the base domains in the body against the company's partners, and alerts the user if there is a discrepancy.
Of course, if BankO or one of its partners has someone inside who can gain access to the computing infrastructure, and then puts a masquerading program at some address inside BankO or a partner, then our method cannot detect this. But such events are not commonly considered phishing at all. Typically, a phisher has no such access into its targets. It does mean that BankO needs to have confidence that its internal network is secure, and likewise for its partners. But it should be doing so anyway, irrespective of phishing. This may also act to constrain the size of a Partner List. Which speeds its lookup by ISPO.
Our method can also be used where the targets are governments. It can be attractive for a phisher to impersonate a national taxation department, for example, and solicit vital information from individuals, like their names and tax ids. Likewise, educational institutions could also benefit. Thus far, phishers have not significantly targeted these. If only because customers of commercial sites probably offer more immediate tangible gains. But for the purposes of finding out important personal information, future phishing might impersonate educational institutions. Plus many such institutions also offer electronic addresses to their students, so they are ISPs in their own right. So from the perspective of being either potential targets or ISPs, educational institutions might use our method.
Previously, we had defined base domains as those which could be purchased. If governments or educational institutions were to use our method, there would be obvious generalizations of base domains to accommodate this usage. For example, in the United States, the federal government agencies use the domain ,gov. We can choose the base domains here to be 2 fields, like ssa.gov or irs.gov. Then each agency might make its Partner List available. Similarly for other governments.
Deployment of our method is straightforward. It can be done on an incremental basis, and quickly. It does not require any rewrite of existing Internet standards.
Consider first ISPs. In most developed countries, something like 5-10 ISPs have over 50% of the market. By necessity, phishers have to preferentially target the users at these ISPs. Perhaps also by choice. The largest ISPs serve a mass market. They are more likely to have technologically unsawy users, who are in turn more likely to be fooled by a phisher. Therefore, a relatively few but crucial set of ISPs have to adopt our method, in order for benefits to happen. But this set can start with just one ISP. Now consider the targets. As discussed earlier, these will tend to be large financial firms, and other large companies that, for example, have many online customers. In total, there are not that many targets. And, like ISPs, they do not have to join en masse for our method to yield
benefits.
An ISP can use the Partner Lists it gets from the companies as input to other antispam methods, to better filter out spam. Plus it might also use these lists to better understand its users' interests and in turn use this to offer better marketing of users and their interests, as discussed in our Earlier Provisionals. (There may be issues here of the ISP having to get permission from the companies whose Partner Lists it is using.) For example, aside from rejecting messages purporting to be from BankO, ISPO might offer BankO guaranteed advertising access to some subset of ISPO's customers. This can be a revenue source for ISPO.
There is also an extra positive side effect of our method. Many spammers forge sender addresses. A spammer might use a false sender address at one of the companies that uses our method, to send a non-phishing message, where the message body has no references to that company, either in the links or text. The spammer is pretending to be from a large company that she hopes will be unlikely to be blacklisted by an ISP. Our method lets the ISP trivially reject this message. Three parties benefit. The user benefits because she gets less spam. The ISP benefits from this, because the user gets a better experience than otherwise, and so hopefully is more likely to stay with that ISP. Plus, if the ISP can reject such non-phishing spam, it also reduces the economics for that spammer, and the chances of more spam coming from that person. Of course, such rejections also reduce the ISP's current storage costs. Finally, the company itself can benefit. The risk is reduced that an ISP using a simplistic method of amassing a blacklist will inadvertently include that company in it, because some spammers are claiming to be from that company. Also, the reputational risk of the company with an end user is reduced. Because some users might buy from a spam message and be dissatisfied with what they get. Unsophisticated users could blame the company, because the message purported to come from it. (Collateral damage.) This is not a trivial consideration, because it tends to be unsophisticated users that purchase from spammers in the first place.
We also include the case where a central website aggregates Partner Lists from various companies and makes these available to message providers, possibly with extra analysis being performed on these lists by the website. Caution is necessary here because such a website would be a natural target for phishers, who may attempt to disable it. As a possible countermeasure, we also include the case where this aggregated information is made available in a distributed fashion across different nodes on the network, and the nodes are managed by one entity. We call this website an Aggregator.
1.3 No Message Authentication
A major advantage of our method is that the ISP does NOT attempt to authenticate a received message, in general. Some other anti-phishing and antispam methods have proposed various means of attempted authentication. But, for example, "Sender ID" (as suggested by Microsoft) is vulnerable to IP spoofing. Plus, as proposed, Sender ID requires extensive changes to DNS servers throughout the Internet and will take several years to implement, even according to its proponents.
Another proposal, "Domain Keys" (as suggested by Yahoo), uses public/private keys to sign outgoing messages. One major problem is that at the receiving end, the ISP has a heavily increased computational and bandwidth burden, to authenticate such messages.
There is a potential trap here for an ISP that tries to authenticate each supposed message from a large company. A phisher can cause an "arms race" to overwhelm the ISP. She makes many new fake messages and transmits them to the ISP at relatively little cost. The ISP then has to authenticate each message. Typically, the computational costs can be significant, especially if strong authentication techniques are used. Bandwidth costs and latency delays can also be significant. Because the ISP may have to transmit some computed information about each message to the company for verification, and wait for the reply. This also creates bandwidth and computational costs for the company, in order to answer the ISP.
By contrast, in our method, the ISP just takes each message as given, and applies our steps. Far quicker to extract body link domains and compare with those in the Partner List. Typically this should be speedier than any authentication technique. Plus, there are no concomitant bandwidth costs and latency delays in communicating with the company or Aggregator. And so that company or Aggregator is not burdened by this.
Of course, our method requires that the Partner List for a company be transmitted in some secure fashion from the company to the ISP. This should be authenticated. But it is realistic to expect that the frequency at which a company's list changes is low. Once per day, say, or even less. (Once per week?) So suppose a transmission is done electronically, with possibly some kind of intensive cryptographic method for authentication or encryption. The amount of computation and bandwidth required is still orders of magnitude less than if the ISP had to do this for each regular message.
Also, as we have alluded to earlier, if a Partner List is short, and it is transmitted electronically, the authentication need not be entirely computational. Suppose the list has just a few domains. Then Dinesh at ISPO rings Laura at BankO after he has received her message. He can recite the domains over the phone, to confirm receipt and authenticity. Here Laura and Dinesh should run some manual checks on each other during the conversation, based on prior knowledge, so that each is confident of the other's identity.
1.4. Phisher Responses
How might the phishers respond to our method? Let J be the set of major ISPs and T be the set of large companies that phishers use for targets. Suppose that a substantial fraction of J and T end up using our method, thus reducing a phisher's revenue. She might then look at smaller ISPs, aiming at these customers who are also customers of T. Or she might try to impersonate smaller companies, aiming at customers of T or of small ISPs. Clearly, even if no other countermeasures are taken against her, her yield will be less.
Plus, the ISPs already using our method for large companies can sign up smaller companies. And smaller ISPs not yet using our method might decide to do so. Especially if there is competitive pressure for them to offer this level of protection to their customers, that their larger competitors already do.
Another alternative for a phisher is social engineering. That is, instead of sending from info@bankO.com, she might send from info@support-bankO.com. But consider this. Currently, without our method being used, over 92% of phishing forges the sender, and only some 2-3% do social engineering, as mentioned earlier. Phishers of their own volition forge senders from the targets because that gives higher yield. Since there is no technical difference in difficulty between the two methods. If you can do one, you can just as easily do the other. So if our method results in a great diminishing of forged senders from the targets, then the phishers shifting to the other approach should greatly reduce their earnings. Plus, we show in Section 2 a method that can attack this approach.
Our method is lightweight and straightforward to apply, for both target companies and ISPs. It mostly requires that an agreed upon query and answer format be promulgated. And thence various large companies and ISPs adopt our method. But, as we have demonstrated, this query and answer format is trivial. It can be quickly deployed.
Our method can greatly reduce losses due to phishing. It would help many people and companies. Language independent and can be deployed globally. When applied, the users do not have to see the phishing messages. So no matter how well worded a message, or how persuasively a phishing web site might mimic a target site, the users do not get fooled. Users do not have to learn new passwords or perform new actions.
1.5 Behavior List
A similar idea to the Partner List is a Behavior List. Written preferably in XML. It can indicate the various types of behavior exhibited by the message. For example, does the
message have popups? And if so, how many? Does it have pop-unders? Does it use scripting?
Earlier, we described the computation of styles. These may be put into the Behavior List.
Section 2. Notphish Tag
We extend the discussion of the previous section into what can be done on the user's computer. We show an objective and computationally lightweight method of detecting both phishing messages and pharms (fake websites). This method essentially treats both equally.
2.1. User Applications
Consider a user, Sarah, operating a browser. For brevity, we assume that the browser is producing visual output that she can see. In other words, she is not visually impaired or blind. Though our methods below can also be applied in the latter cases, where the browser might produce aural or tactile output to her.
Our methods here are not restricted to a browser. They can be used in any user application that can display electronic messages or web pages or which gets or manipulates data that comes over a network, and which permits the addition of third party plug-ins. Plus, if such a program did not, our methods below still can apply, if the program is reimplemented to permit the addition of our plug-in, or, equivalently, of its functionality.
Also, we use the common and important example of the Internet, for our network. But in general, our methods apply to any digital communications network, including a phone network.
Sarah is looking at a network address, like an URL. She might be reading an email message, or, more generally, viewing a web site's document. This document may claim to be from a bank, BankO. It may also have links to other URLs. Is the document really from BankO? Hitherto, most attempts to answer this question have used some authentication or encryption method. For example, it might involve the document having a hash signature computed from the rest of the document, and signed with the bank's private key. The viewing program (browser) might then take the hash, go out on the net to the bank's website and find its public key. Then, the program would apply the key to the signed hash. The result could be compared to the hash it directly makes from the document. If these match, then it considers the document as having been issued by the bank.
Many variants are possible. But the example indicates the general flavor. These techniques are heavyweight. Authentication, especially with public and private keys, is computationally intensive.
We offer a simpler, quicker approach. If the document is indeed from BankO, it can have a tag. The name of the tag is fixed. It only needs to be different from the names of the standard tags in HTML. Plus, it should also be different from the names of non-standard but common tags supported by major browsers, like Internet Explorer or Netscape. We choose to name the tag "Notphish", because of the descriptive value of this name, though of course any other name could have been chosen, subject to the above constraint.
The tag can have an arbitrary number of variable=value pairs. Specifically, it has at least one such pair. The meaning of this variable is that it designates the value as some kind of network address, similar to, or the same as, a URL or URI (Universal Resource Indicator) or phone number, for the bank. The name of the variable is arbitrary, but let us call it "a". So we might have a tag like :
<notphish a="bank0.com" />
One of the rules of any browser is that if it encounters an unknown tag like the above, then it does not display the tag. So any documents using the tag can be viewed in a browser with no visible change or degradation.
In the above tag, we make the following optional but preferred requirements. None are essential to our method, but they have the advantage of making parsing easier to program and quicker to run.
1. The tag name and variable name are always in lowercase. HTML is deliberately permissive, so that, for example, these are the same to a browser: <body>, <BODY>, <Body>. But our requirement for our tag means that parsing does not have to first push an input string to lower case, for example, before looking for "<notphish ". 2. The tag is terminated with these characters, and in this order -{space, forward slash, greater than} . The inclusion of the forward slash is characteristic of XML tags, which are more rigorously formatted than HTML tags.
3. The value of the variable is bracketed by double quotes.
4. There is no whitespace (which is {space, tab, newline}) before or after the equals sign.
5. The only character between "notphish" and "a" is a space character.
Suppose now the browser has a plug-in program whose job is to handle this tag. Optionally, but preferably, the plug-in runs whenever a page is loaded, but before it is displayed. Also, the plug-in has an optional but preferred button, that is visible. Though Sarah may be able to suppress the display of this plug-in's button, just like that of any other plug-in's visible component, if she desires, and if the browser permits such an action.
Plus, optionally but very preferably, the plug-in tells the browser that no scripting routine
executed from the page should be able to launch a window over the plug-in's visible component.
By default, the plug-in parses the current page. It looks for such a tag, as described above. If the tag exists, then it extracts the address's value. It also extracts any links from the page. Usually these might be outgoing links. But they might also be incoming links. (The latter might be when an HTML message loads an image from some URL.)
The link extraction may possibly involve evaluation or attempted evaluation of scripting routines, if the page contains dynamic hyperlinks. (See Section 2.4.)
The plug-in reduces these links to their base domains. While doing so, it optionally, but preferably, makes a list of the different protocols used by the links. Typically, this list might be {"http", "https","ftps"...}. The point here is that a bank might require that authentic links only use a secure protocol like https.
The plug-in can then go out to an Aggregator Web Service, as mentioned in the previous section. This is a network site with a list of keys. Each key is the base domain of an Aggregator customer, like a bank. For each key, the Aggregator has its Partner List. The plug-in sends the address value of "bankO.com". The format of the message is arbitrary. But let us write it in XML format as -
<a>bank0.com</a>
Notice that the name of the tag here, "a", does not have to be the same as that of the variable in our tag in the web page. A short name is good, in order to reduce the size of the message that goes on the net. While the network bandwidth reduction is trivial at the client end, it reduces the incoming bandwidth at the Aggregator. It also makes the parsing quicker. Both factors help the Aggregator scale to heavier loads.
Optionally, the plug-in also sends a list of the unique base domains and the list of protocols for these domains. If so, then the format of this is arbitrary. But we could choose to write it as -
<a>bank0.com
<p>auto 15.com.au</p>
<p>store23.com</p>
<p>store.com.cn</p>
<t>https</t> <t>http</t>
</a>
In the above, the plug-in is saying that for any of those domains, a link was found that used http or https. Optionally, a more precise formulation might be to specify for each domain, the protocols found for its links. So we might have -
<a>bank0.com <p>auto 15.com.au
<t>https</t> </p>
<p>store23.com
<t>http</t>
<t>https</t> </p> <p>store.com.cn
<t>https</t> </p> </a>
In the above, it says that there were at least two links to store23.com, and one of them
used http and the other used https. Whereas for store.com.cn, there were one or more links, all using https.
The partner and protocol tags have short names. Note that purely as a matter of visual convenience, we wrote the above examples across several lines. In an actual message, to further reduce bandwidth, the newline characters could be omitted.
The Aggregator checks if it has bankO.com as one of its keys. If not, then it returns a negative result. The format of this is arbitrary, but let us write it as "<no/>".
But suppose the Aggregator has a key of "bankO.com". If it has not gotten a list of partners from the plug-in, then it replies with bankO's official Partner List and, optionally, the permissible protocols for links to those partners. This format is arbitrary, but preferably it should follow the format of the previous example of the plug-in sending partners to the Aggregator. Thus, the Aggregator might reply with -
<a>bank0.com <p>auto 15. com . au</p> <p>store23.com</p> <p>store.com.cn</p>
<t>https</t> </a>
The plug-in then compares every base domain found in the web page with those in the official Partner List. If all the domains are official partners, and the protocols are the approved protocols, then the plug-in can show this in some fashion. Visually, it can show a color or image that indicates a validated page. Otherwise, it shows a different color or image for a non-validated page. Optionally there might only be two different colors, or two different images, essentially meaning Yes or No. As simple as possible.
Suppose we have determined that a page is non-valid, as in the previous paragraph. We define it as being "invalid". This is quite distinct from a page being non-valid simply because it lacks our tag, which will be true of most pages.
Optionally, for an invalid page, the plug-in can turn off all links in the page, or only the invalid links. Plus, the plug-in might highlight or in some way change the representation of the invalid links. It could pop up a window warning the user about the page. Very useful in alerting Sarah about a possible phishing web page or message.
If this behavior can occur, there can also be the means for letting Sarah turn on or off these actions for future such situations, or for the current page. The latter case is useful if Sarah decides that she really wants to follow a link that has been disabled. So she is able to enable the link.
For an invalid page, optionally, the plug-in might send the page, or some subset thereof, to one or more of the bank, Aggregator or appropriate regulatory authorities.
One key advantage of this method is that Sarah does not need to do anything different in her browsing. Unless of course the plug-in detects an invalid page. To her, the results of our methods can be as intuitive as being able to recognize a hyperlink. Above all, she does not have to manually try to discern if a given page or message is perhaps phishing. It has been realized that many phishing pages are virtually indistinguishable from a real bank's pages. So the manual inspection can be very error prone and subjective. Another reason that this is bad is that it leads Sarah into reading the text. Which is usually very well crafted by the phisher to alarm and mislead her.
Plus, Sarah does not have to examine the HTML source code. Given the complexity of many web pages, this can be very confusing to many people, who are often unfamiliar with HTML to begin with. Some other antiphishing methods have recommended this user inspection of the visuals and the HTML.
Our method provides an objective answer, and reduces the chance of the user being fooled.
Alternatively, if the plug-in sent a list of domains and protocols from the page, then the Aggregator might perform the previous test. If all the page's domains and protocols are in the official lists, then it might reply with "<yes/>", for example. While if any domain or protocol is not in the official lists, then it might reply with "<no/>", for example. The plug-in can take this reply and respond accordingly to Sarah.
By default, the plug-in might be configured to automatically perform this on every page viewed by the browser. The computational and bandwidth requirements are minimal. Most pages will not have the tag, so the plug-in will not need to consume bandwidth to ask the Aggregator. Plus, when the user uses the browser to go from one page to another, that change of page occurs on the user's timescale. Given current computers, the time for the plug-in to look for the tag and, if it exists, doing the other tasks, will usually be negligible on that timescale.
But Sarah can also decide to turn off this default behavior. Instead, she might want to invoke the plug-in explicitly. This could be done if, for example, the plug-in has a visual representation in the browser as a button. Then Sarah can press it to run the above method on the current web page.
Note that the network communication to and from the plug-in is optionally but preferably unencrypted plain text. A vital saving of computations on the browser and on the other side of the connection.
The plug-in can also cache data it gets from the Aggregator. So that in future, it would look first in its cache, and, if the desired data is not present, then it asks the Aggregator.
Our method is a significant generalization of the previous section. We can now apply antiphishing methods not just to electronic messages, but to any web document. And do both in essentially the same fashion.
2.2 Extra Tag Information
More information can be stored in the tag. A variable could be stored that indicates to the plug-in whether it should upload the base domain of the page, or the full address of the page, or omit it. For example, we might call such a variable "u", with possible values of "1", "2" and "3", where these correspond respectively to those different choices. Then we might see a tag like this:
<notphish a="bank0.com" u- ' 1 " />
We might also choose to save space by adopting the convention that when no such address information is to be uploaded, then this variable and its value be omitted. This gives us back the earlier case.
The format for uploading the full or partial address is arbitrary, but an example might be as follows, where for simplicity, we assume that the plug-in is not uploading any other information:
<a>bank0.com <u>K/u>
<v>someotherplace . com</v>
</a>
By sending address information to the bank or Aggregator, the plug-in gives more information to the latter party or parties. For example, it can be used to detect if a page
has been copied from some authorized web site to an unauthorized web site. This is important because some phishers set up pharms, which are web sites that pretend to be those of banks, say, in order to trick a visitor in revealing her account information. These pharms might have several pages copied more or less verbatim from the real bank pages.
Plus, if BankO were to require U=1T1, say, then the absence of any uploaded address information received by the bank or the Aggregator can be taken as sufficient reason to reply to the plug-in that the page is invalid.
Another variable that could be stored is an id for the page or message. This id could be some string, as in this example:
<notphish a="bank0.com" id=" 12345asdfghj " />
If such an id is detected by the plug-in, then it could upload the id. The format for doing so is arbitrary, but an example might be as follows, where for simplicity, we assume that the plug-in is not uploading any other information:
<a>bank0.com <id>12345asdfghj</id>
</a>
The id enables a useful new capability. Imagine that the message or web page being viewed is an ad. By associating an id with this ad, the bank can then restrict what partner links are on this page. Suppose BankO has the partners given earlier - autol5.com.au, store23.com, store.com.cn. It gets an id sent to it of "jld". This maps to a page that should only link to store23.com. So it sends only that partner back, in its reply. The plug-in gets this reply and compares it to the base domains extracted from its page. If these are autol5.com.au and store23.com, then the plug-in marks the page as invalid. Without the id, BankO has to send back its entire Partner List. Whereas now, it has more precise
control. Possibly for targeted marketing.
It is important to note that the id does not give the phisher any advantage. Suppose, for example, that she copies a valid page, with such a tag, to a web site she controls. She edits the page. Whether she changes it to an id valid for another page or message, or to a non-existent id, or leaves the id unchanged, any Partner List at BankO or the aggregator will still not include her domains.
In general, an instance of the tag might also include variables for both uploading and id.
2.3 Major ISPs
A very common usage of browsers is to read electronic messages. Each major ISP has its own format for displaying a message in a web page. Typically, aside from the body and header display, there are ancillary regions of the web page. Often used to navigate between messages, and related tasks. Plus, the ISP might put clickable ads in these regions. If the message has our tag, we don't want the domains or protocols in these ads, or those found from any other links outside the body, to be compared with the bank's Partner List. The plug-in can have logic that is aware of each major ISP's message displaying. In Section 3, we explain in greater detail how this is performed.
2.4 Aggregator
Here we expand on the Aggregator's role. Suppose the plug-in sees a Notphish tag that points to bankO.com, and the plug-in has never gone there before. In general, it cannot go directly to bankO.com and get its Partner List and protocols. Because this lets the phisher use a variant spelling on a bank's domain, and then run a server at that variant address to furnish a false Partner List. For example, a phisher might buy a domain of bank01.com,
and establish a web server on it, with a Partner List. Then she sends out messages claiming in the visible text to be from bankO, with a tag that points to bank01.com.
So the first time that the plug-in sees a Notphish tag pointing to bankO.com, it might need to seek out the Aggregator, as described earlier. But now that it knows that bankO.com is a valid domain, as attested by the Aggregator, it may optionally store bankO.com in a list of valid domains. This might be only in volatile memory, for that instance of the browser. Or it might store the list on the disk, for future invocations of the browser.
Then, whenever it sees a Notphish tag, it can check that address first against its local list of valid domains. If the address is present, then the plug-in might go to that address directly, to ask for its Partner List. While, if the address is not in its local list, it proceeds as before to ask the Aggregator.
If the list is committed to disk, then to prevent it growing unbounded over time, the plug-in might have logic to prevent it. This includes, but is not limited to, the following:
1. Let the user manually erase the list. Note that the user might not be permitted to edit portions of the list. It then becomes trivial to write false data. So another user of the browser might then be fooled by this. Imagine a browser in a library or
Internet cafe, and one can readily see the danger. While this only affects that particular browser, we should avoid it anyway. It also means that the list should be written and read in some encrypted form.
2. With each address, store the time of its last access. Then, periodically, purge the outdated entries.
3. An Aggregator can invalidate a domain.
Another optimization is that when the plug-in gets the data for BankO, it might store these in memory or disk. It might choose to consult this, when it encounters future pages with tags pointing to BankO, instead of asking bankO.com or the Aggregator. If so, it should
take into account any starting or expiration dates for a partner. So that if the current time for a partner is after an expiration date, for example, then the plug-in needs to go out on the net and get an updated Partner List.
Another possible role of the Aggregator is to get some measure of how heavily the banks are using our methods. Clearly, whenever it gets a query about bankO, it can increment a counter for that bank. But suppose the plug-in were only to ask about bankO once, and then later query bankO directly? The Aggregator can record the plug-in's address in a list associated with bankO. The address might be the computer's network address or hardware address.
There is also the possibility that the plug-in need never contact the Aggregator. This could arise if when the plug-in is installed, it comes with a hardwired list of known banks. Then, so long as the browser stays within those banks, there is no need to ask an Aggregator about a different domain. While this may seem overly restrictive on the usage of the browser, imagine a plug-in that expressly restricts its validating steps to that list. That is, if the plug-in comes across a page with a Notphish tag pointing to a domain not on the list, the plug-in does not attempt any validation. Nor does it indicate that the page is invalid. The point here is that some combination of the author of the plug-in and whoever paid for the plug-in's distribution, can restrict its validating utility to that initial list. (Who presumably might have paid the former party or parties.)
A simple extension is to have the plug-in be able to accept updates to the list, or also deletions from the list, across the network. This would probably necessitate some type of authentication mechanism, in order for the plug-in to be assured of receiving appropriate data. Regardless of any particular mechanism, this remote updating would let the plug-in owner charge others, to be put into the updates, or not to be removed from the plug-in's list.
2.5 Dynamic Hyperlinks
In Provisional 60521698, we discussed the concept of dynamic hyperlinks. It is possible for a web page to contain a hyperlink, where the actual link is found from a scripting routine. There are several possibilities for what the plug-in might decide to do, if it encountered one of these. The most prudent is to refuse to validate the web page, without even attempting to evaluate the routine to find the link.
Or, it can try to do the latter, using the ideas in that Provisional. If successful, then it can use any such base domains found in this fashion, and proceed with the other steps in this Invention.
2.6 Plug-in Distribution
An important issue is how the plug-in gets disseminated to the various browsers. It is tempting to make it available from a central web site, like the Aggregator's. This is possible. But a danger is that phishers could offer false plug-ins via web sites that are perhaps similarly named to the Aggregator.
An alternative method is for the plug-in to be physically distributed on media, like a CD. This might be financed by one or more of the banks that are registered with the Aggregator. The CDs might go to customers of the banks. The cost of this, and the knowledge of the customers' addresses, is deliberately used to stymie the phishers, who are unlikely to be able to afford the former, or know the latter. Also, even if a phisher could overcome both objections, it would be difficult to hide the physical mailing of many thousands of (fake) CDs.
A related issue is what happens when a phisher writes a fake plug-in and somehow gets it disseminated to a browser. We have a countermeasure where the user manually types in the address of the Aggregator into the browser's address panel, and directs the browser there. The Aggregator can have a web page where the user can upload the plug-in. (This
may not be possible in some browsers.) Then, the uploaded binary can be compared with the real one.
2.7 ISP Methods
We assume that we are an ISP that is using our method in the previous Section, and have a list of target companies, and for each such company, a Partner List. Consider our address to be ispO.com. In what follows, and without loss of generality, we assume that the target companies are banks. Define Gamma as the list of target domains. Define BankO as one of the targets, with its base domain in Gamma as being bankO.com.
Consider an incoming message. Our methods here concentrate on determining an appropriate base domain for the sender of the message. We then take this domain and see if it is in Gamma. If so, then we apply the steps in the previous Section by finding the domains in hyperlinks in the message body and comparing these to the domain's approved partners. So, for example, if the sender information said info@help.bankO.com, we would find from this the base domain of bankO.com, and then see that it is in Gamma, and thence apply the earlier steps.
But a phisher, Jane, can construct more elaborate attacks. One possibility is that she might write a scripting routine. This can run by the browser if the user [recipient] is viewing the message in a browser, and if she has enabled it to run scripting code present in messages. Currently, the de facto scripting language for use in browsers is JavaScript, but our remarks apply to any scripting language supported by a browser. The combination of a scripting language and browser may enable the following: The routine might be able to display an image, over the portion of the browser that shows the sender information. The image might say, for example, info@bankO.com, and the body of the message might have text claiming to be from BankO, but with a link to Jane's domain. The catch here is that the sender field in the message header might have a base domain unrelated to bankO.com. Jane can essentially write any address she wishes here, except
one from bankO.com.
A variant of this attack has already been seen ("Phishing Technique Replaces Web Browser Address Bar with Malicious JavaScript Fake", http://www.antiphishing.org/news/03-3 l-04_Alert-FakeAddressBar.htmQ where phishers used JavaScript to display an image over the Address (URL) portion of the browser. It is technically harder to have a routine that can show an image over the sender (From:) portion of a displayed message. But not impossible. The extra difficulty arises because the sender information is written inside a web page, and exactly where within this varies with the choice of the recipient's ISP. Each ISP may have a different program that shows the sender information in different locations in a page. The easiest way to see this is to have accounts at two different (and large) ISPs, like Yahoo and Hotmail, for example. Then, look at a message under Yahoo and see where the sender is displayed. In general, this is at a different position than where Hotmail will show the sender.
Whereas the Address is always shown in a fixed location, given a particular browser. So in the latter case, Jane has an easier problem. But Jane can still adapt. Remember that we are assumed to be an ISP. Jane knows that she is sending to a set of users at our address. And we have to assume that she knows where our display software will be showing the sender. There is no way to prevent her knowing this, since it is visible to any of our users, and she can sign up as a user beforehand, possibly under an assumed name. Hence Jane can customize her routine for each major ISP on her mailing list, so that the routine can correctly position an image over the sender line.
Of course, if the user at an ISP forwards mail to another ISP, then the positioning of the image may be wrong. But Jane may be willing to accept this, under the possibly reasonable assumption that most users don't forward their messages. Especially if they are using a free messaging service. Most of those do not permit forwarding, unless the user signs up for a premium, paid service.
Also, if the scripting language and the browser in which it runs are sophisticated enough, then the script may be able to deduce what ISP is actually presenting the message to the browser (in some markup language like HTML). Hence, Jane may be able to write tests into her script that can correctly position an image, for the major ISPs. And so, she can largely handle the forwarding issue.
To counteract the image, we have several responses. We can detect a scripting routine that will put an image up over the sender line. If so, then without further ado, we might immediately regard the message as spam or phishing, and treat it with whatever existing methods we have in place for such messages.
Alternatively, we can attempt to programmatically deduce what the address in the image. While we have called this an image, in general, it is a window that might show an actual image, or it might show graphical objects (commonly known as widgets) that display text. In either case, we can try to find the address. For an actual image, this should be easier than the general case of trying to discern a set of text that is displayed in a way deliberately intended to defeat machine recognition. These are commonly used in challenge-response situations, where the entity issuing the challenge wants the correct response to be manually entered by a human. In our situation, Jane is severely constrained, because the image of text that she makes must look like an authentic sender image. Typically, this image might show black text against a white background, and the text is presented in a simple and standard font. Hence, we should be able to apply optical character recognition techniques to the image, with high precision.
If the script is showing widgets, it is possible that the image of a sender address is generated dynamically. That is, suppose the image is seen by a human as "info@bankO.com". This text might not actually exist as a single string inside the script. Instead, there might be arbitrarily complicated code that builds this string. This is to avoid us being able to simply parse out an address. We discussed a similar method that can be used by spammers, in Provisional 60521698. There, the spammer was making a
dynamic hyperlink in the message body. Here, it is not a hyperlink that is being made, but the dynamic aspect is the same. Likewise, we can evaluate the script in an isolated "sandbox", to try to discern the address it constructs. The methods of that Provisional can be applied, especially if Jane decides to write infinite loops into her script, to stymie our evaluations. (If in fact we detect an infinite loop in the script, then we cannot evaluate the message further. So we prevent the message from going to its recipient and we exit these steps.)
If successful at extracting an address from the image, we find a base domain from it. We then use this base domain, instead of that written in the message header's sender, as the sender when applying our earlier Provisional.
Optionally, we can also set a heuristic flag, which we term a style, to indicate that the message has an image over the display of the sender field.
We also generalize this idea of attacking scripting that makes an image that covers up part of the browser. Thus far, we discussed an image blocking the sender field in the browser, for a particular combination of browser and ISP. In general, the scripting might be able to bring up one or more images that block any portions of the displayed message. Plus, the actual sender might not necessarily be a phisher. She could be a generic spammer.
Optionally, we can set one or more style flags, if we detect scripting that makes these images. In general, the more of these flags that are set, the more suspect a message is. Some of these flags might depend on whether an image blocks a particular part of the displayed message. An important example is where an image blocks the subject line, replacing it with another subject line, unbeknownst to the user. Why would Jane do this?
Early antispam techniques often tried to record a set of subject lines from messages that were considered to be spam. Or, possibly in conjunction, a list of keywords was built
from these messages. Then, future incoming messages had their subjects compared with these lists. If there was a match, then a new message was classified as spam. But spammers soon learnt to introduce unique variations into the subject line. Often by introducing deliberate misspellings, where the user could still discern the original words.
Being able to bring up an image of another subject gives a spammer even more flexibility. For example, the image's subject might not have any misspellings. To give it a more professional appearance of not being spam.
In particular, changing the appearance of the subject line has attraction to a phisher like Jane. Unlike a generic spammer, Jane has relatively little leeway to introduce any sort of visible randomness into her message. It must appear to be as reputable as possible. So the simple antispam technique of checking the subject line can be more of a danger to her than to a generic spammer. Therefore, she has incentive to make a subject image.
In passing, we address one issue that might appear to simply prevent, not these images, but the ability of them to mislead the user. Imagine she is logged into to her ISP and looking at a list of her messages, as opposed to looking at the contents of a single message. Typically, such a list shows the subject line and sender for each message. These are found from the header field and not from any scripts that might be run when she looks at a given message. Hence, if she goes from the list of messages to a message, she should be able to notice any discrepancy. Thus invalidating any images and alerting her that the message is a fake? Not necessarily.
Firstly, she might not notice any discrepancy. It depends how carefully she remembers the information in the directory for that message. Remember that Jane is aiming at inexperienced and naive (and careless) users. They are far more likely not to notice any discrepancy.
Secondly, a common way of viewing messages is to read a message, then go directly to
the next message in the list, in ascending or descending order. In doing so, the user unlikely to notice any discrepancy between what is presented before her in a message, compared to what she might have seen for that message in a far earlier perusal of the list of messages, especially if she has many messages.
2.8 Similar Names
Suppose we have now found a base domain for the sender. (Here, "we" might be the plug-in or the ISP.) This may have involved using the steps in the previous section, if Jane made a scripting image over the sender field. Otherwise, we found it from the sender portion of the header.
There were 1422 unique phishing attacks reported in June 2004. Of these, at least 92% used forged sender addresses. That is, the base domains were those of actual targets (usually banks). Some 2-3% used "social engineering", where the sender domain was a real domain with a name close to that of the target. For example, for the Visa credit card organization, one sender address used was support@verify-visa.org. The domain in this instance had no affiliation with Visa. ["Phishing Attack Trends Report", June 2004, by the Anti-Phishing Working Group.]
By applying the steps in Section 1, we can detect when phishers forge sender addresses at banks, if the banks have Partner Lists, and the phisher puts a link to a domain outside a bank's Partner List. So suppose the current 2-3% that use social engineering become the majority (~ 100%) of new phishing messages. These are harder to fool people with.
Also, we can also perform a variant of the steps in Section 1, when we are considering a given message. Gamma is our list of target banks. For each entry in Gamma, take the leftmost field. This is the key field which designates that bank. So, if we have bank0.com, we now have bankO. Or, if we had bank0.com.au, we now have bankO. Then, see if this field exists as a substring of the base domain of the sender field in the message. If so,
then possibly we have a phishing message using a similar domain. Then we apply the steps in the previous Section to the message.
Thus, for example, if the sender base domain was info-bankO.com or abankO.com or bankO-help.com or help-bankO.com, we would interpret this as attempting to mimic bankO. Hence we apply our earlier steps. So that we would now ask the Aggregator for the Partner List for info-bankO.com. If this did not exist, then we make the message as invalid.
We can also apply another test. Given each target in Gamma, we can precalculate an accompanying list of possible, deliberate misspellings. For example, if a name has the digit zero 1O', replace it by V. So bankO gives us banko. Of if a name has the letter T (ell), replace it by '1' (one). While these rules can be applied programmatically, we can also have variant misspellings that are found manually.
Thus, every target in Gamma has an optional list of misspellings. We can then apply these to the sender base domain, to detect possible phishing. If we get a match, then apply the steps in Section 1.
We can also apply this in conjunction with our previous Section. So for example, suppose for bankO, we have its misspelling list containing banko. Then, suppose we got a message with a base domain of info-banko.com. We might consider this to be pretending to be from bankO, and apply the steps in Section 1.
It can be seen that the phisher going to this extreme reduces the chances of fooling the bank's customers. The misspelling list for a given bank obviously can have some subjectivity in its construction. But listing the obvious misspellings forces her to choose far more blatant misspellings, that the recipient is more likely to detect.
Our methods also fill a gap in existing methods by others that search domain name
registrations for names similar to those of banks or large companies. In our example of bankO, those methods would detect a registration like secure-bankO.com. Typically, several things might happen. BankO might be informed of this. And if that domain is instantiated at an actual IP address, its web pages (if any) might be crawled in order to see if the author is trying to mimic BankO's web pages. This is done for phishing where the actual phishing link in the message would point to secure-bankO.com. These methods are useful, but only apply when the phisher will maintain a website at that domain. They are moot when the similar name does not exist as an actual domain, but simply as a fake domain in the sender line.
Another advantage of forcing phishers towards social engineering of the sender is that banks can reduce their losses in another way. Currently, most of the financial cost is borne by banks. In part, this appears to be a public relations policy, wherein a bank might voluntarily indemnify a customer who got fooled and suffered losses. But if Jane now has to pretend to be info@support-banko.com, say, and then in the message body, she pretends to be from bankO, then the bank might reasonably argue that it does not need to indemnify anyone who got fooled by this, if the sender did not even pretend to be from the bank. Purely from a public relations viewpoint, it is now in a stronger position.
2.8 Phishing via a Partner
If our example of BankO uses a Partner List, then there is a possibility that Jane might launch an indirect attack, using a partner. Suppose the list has markO.com, and Jane sends a message pretending to be from it. If markO has a Partner List of its own, then if Jane has a link to a domain not on markO's list, we can catch the message, without recourse to BankO's list.
We have to assume that Jane can find out that markO is a partner in the first place, because BankO probably sent out many messages that pointed to it. Under these circumstances, there is no effective way to hide markO's status from Jane. Nor should
there be. But not only does this tell her about markO; as an element of psychology, it implicitly validates markO to BankO's recipients. That is, a message purported to be from markO, and which mentions BankO, might have credibility to an unsuspecting user.
Now suppose markO does not have a Partner List. Jane sends a message, claiming to be from info@markO.com. It has one or more outgoing links to bankO.com. Or one or more incoming links from bankO.com, where it is loading images from bankO's website, say. Optionally, it might also have incoming or outgoing links to markO.com. All these are fine. But the message also has an outgoing link to somewhere.com, for example. This might be pointing to Jane's website. How do we test for this?
Several ways. Firstly, BankO might choose not to have any other companies in its Partner List, except possibly subsidiaries or parent companies. And if so, those entities all have Partner Lists that are consistent with what messages will be sent. This would prevent any links to outside companies in its messages, but it may regard that as acceptable.
Secondly, BankO might require that its partners also have Partner Lists at the major ISPs or Aggregators. If our method gets widely used, then this may not be an undue restriction, though it may still limit BankO's partners to major companies. BankO may or may not see this as a handicap to its marketing.
Thirdly, BankO might require that if a partner does not have a Partner List at the major ISPs, then the partner explicitly makes a Partner List and sends it to BankO. Call it Rho(p,bank0.com); where it is a function of the variable p=a specific partner and of the bank. BankO vets these entries to see if they are acceptable, by whatever criteria it chooses. Rho(p,bank0.com) means that for any messages coming from p, and which have links to or from BankO, then the only valid outgoing links are those in Rho.
Then, assuming Rho is acceptable, BankO sends these to the ISPs or the Aggregator. Later, when we (the ISP) get a message claiming to be from markO.com, and which has
links to bankO.com and other addresses, we check that these are all in Rho(mark0.com,bank0.com). If not, then we designate the message as phishing.
In passing, note that the totality of such Partner Lists amassed at ISPO can be used to build or enhance a social network. That is, if BankO has Store 1 in its Partner List, and
Store 1 has a Partner List, containing Car5, then we can imagine these as nodes in a graph, with BankO connected to Storel connected to Car5. Currently, there are many methods of building social networks from electronic messages. A major problem is assessing the strength of a connection between two nodes. Here, the Partner Lists offer high value information.
2.9 Partner List and a Redirector
Sometimes, a bank may choose to have a redirector on its website. Probably for use by its own web pages or by its partners. The danger is if the redirector will redirect a query to an arbitrary URL given in the query. A phisher can send messages, claiming to be from the bank. AU the links in the message are to the bank, so everything appears correct. But one link is to the redirector, telling it to redirect to the pharm website.
To combat this, we offer two possibilities. Firstly, the bank should check the base domain in the destination of any redirector query against a list of valid destinations. This could be a separate list, analogous to the Partner List, which hitherto has been for valid destinations in outgoing messages from the bank. But the bank may find it convenient to also use the Partner List for redirections. Hitherto, we have discussed only an ISP using the bank's Partner List.
Secondly, consider ISPO that has the bank's Partner List. When it gets a message claiming to be from the bank, it also checks any links to the bank. If a link has a redirector, then ISPO checks the destination's base domain against the Partner List.
2.10 Extending to Persons and Smaller Companies
We have focused mainly on large financial companies as the targets of phishers. But smaller companies and even individuals might also wish to use our methods. Why? For one thing, if our method is widely used by large companies and ISPs, it might successfully deny those those companies as targets to phishers. Who might respond by gravitating towards targeting smaller companies. The latter can then protect themselves by using our methods and furnishing Partner Lists to the ISPs.
But our methods can also be used as a cheap, competitive means of validating electronic messages, compared to methods that use full authentication.
The latter methods authenticate each message. They offer some way for the recipient to verify that the message was indeed sent by the purported sender, and that it was unaltered in transit. But by their very nature, they are computationally intensive.
Whether or not the computations are performed by the recipient's computer or ISP. Often, there might also be a bandwidth and latency cost. Because the recipient's computer or ISP may have to contact the sender's computer or ISP, or a key handling authority, to obtain the sender's public key, if the method involves public/private key techniques. So this will consume some outgoing and incoming bandwidth. Perhaps just as importantly, the latency can be mostly the time spent waiting for the other computer to process the query.
In contrast, our methods do not take as much computations per message. Searching for hyperlinks in a message body, reducing these to base domains and comparing to a
Partner List should usually be faster than any secure authentication method. Plus, there is no ancillary bandwidth and latency cost, because all this can be done on the ISP's computers, or by the plug-in on the user's computer.
2.11 Further Extensions
Our methods usually validate the links in a message, and not the text. But, as phishers have shown, the links in a message can be the most important parts of a message, as far as financial attacks are concerned. Thus, an ISP could expand the scope of usage of our methods by offering them as a cheaper alternative to authentication, for any person or company. Plus, these persons or companies need not be traditional customers of the ISP. That is, they need not be getting messages at the ISP.
Furthermore, as a trivial extension of the steps in Sections 2.1 and 2.2, the Notphish tag could have an argument that was the hash (or some other type of signature) of the message or web page. Here, we could envisage that the hash was done with the Notphish tag absent. Then, the tag is inserted into the web page or outgoing message, with the hash included. Hence, if the plug-in saw the Notphish tag, it might remove the tag, compute the hash of the remaining message or page, and compare with the tag's hash value. If the two disagree, then the message or page is invalid. If the two agree, then the plug-in should still consult the Aggregator, which has presumably been furnished with valid hashes of messages or pages, by the banks that wrote them.
Related to the previous paragraph is a simplified procedure. Where the Notphish tag does not have a hash. But it has an field that says find the hash. Thus the plug-in finds the hash of the message or page, and then compares this with valid hashes from the Aggregator.
The steps in the last two paragraphs let a bank lock down, not just the links in its outgoing messages, but also the text of them.
A simple extension of this method is for the ability to do multiple hashes per message or page. The author could embed begin and end tags for each part that is to be hashed. Preferably, these parts do not overlap, for simplicity. The names of these tags could be some agreed upon standard. Whence the plug-in could find the tags and do the hashing. Then it would contact the Aggregator and compare with the hashes that the author's
company had uploaded to the Aggregator.
Another variant deals with the case when a message has attachments, where these attachments might not be in plain text or HTML. The author might hash the attachments, using one hash per attachment, say. Then, these could be uploaded to the Aggregator. Plus, the Notphish tag that is put into the plain text or HTML attachment could have an argument that indicates to a message provider or plug-in that gets the message, that hashes should be made of the other attachments.
Another variant deals with the presence or absence of scripting. For example, the Notphish tag might have an argument that indicated that certain types of scripting should not be in the message or page.
In the above, when we described a method that was used on the client side browser, for example, it may also be used on the ISP side. Or vice versa. For example, an ISP might offer a premium service to its users, whereby it checks a user's messages. For those with the Notphish tag, it performs the validation steps described above, that the plug-in did. If the ISP finds that a message is invalid, it might delete the message. Or make it available to the user, but in a separate folder (called "Invalid" perhaps), and possibly with the links turned off. This is useful if some of its users do not have the plug-in, and so cannot do the steps of this Invention.
In general, any method might be used on either side. Though any given method might perhaps have greatest efficacy on a particular side.
Section 3. Finding Message Bodies
In the previous Sections, we discussed various means of attacking phishing in electronic messages, and also in web pages. Here, we expand upon the method in Section 2 of reading
electronic messages in web pages.
Our methods here can be used to validate electronic messages that are read in a browser. The validate includes, but is not limited to, the detection of phishing messages.
3.1 Finding Message Body Delimiters
In Section 2, a salient point was how a plug-in that runs in the browser can extract the body of a message from a web page. The problem is that suppose the web page is displayed in
HTML. We cannot simply find the <body> and </body> tags, and then extract everything in between. These tags refer to the body of the entire page. But when an ISP displays a message, the actual body of the message itself is embedded in some subset of those tags. In general, outside that message body, but still within the overall <body> and </body> tags, the ISP puts other information, including hyperlinks. Some of these links are to itself, so that the user can go to the next and previous messages, for example. But some links may be to third parties. The ISP may be showing ads for those parties.
In applying the methods of the previous Sections to a general web page, we extracted all hyperlinks. But for the important case of viewing electronic messages supplied by an ISP, we only want the links inside the message body.
The plug-in can find from the browser the current URL that the browser is showing. From this, it can reduce the address to a base domain and then compare it to a list of known ISPs. For each ISP on this list, the plug-in can have logic that knows of the specific delimiters used by that ISP. Most often, these delimiters might be implemented as tags. The tags may or may not be comment tags.
Also, if the tags are not comment tags, then the end tag need not necessarily have the same name as the start tag, preceded by a forward slash. That is, suppose the start tag is
<messageBody>. Then, following a common XML convention, the end tag would be written as </messageBody>. But this is not necessary. The end tag might be <endMessage>, for example.
It is also possible that an ISP might choose the start and end delimiters to be the same.
Plus, a given ISP's delimiter tags need not have names that are unique to that ISP. For example, if the tags are non-comment tags, then the ISP might choose an existing HTML tag, and insert an extra variable=value attribute, to make a delimiter.
An ISP might inform the company that wrote the plug-in about the exact delimiters that the ISP is using. Or the company might periodically query it. Then, the plug-in might periodically query the company for such information. Or, the company might update the plug-in, for such plug-ins that it knows their network addresses, and that are currently running at that address and can receive such updated information.
Another optional method is for the plug-in to have a list of ISPs, and occasionally, it queries one or more of these, asking for the current delimiters. The plug-in can cache these, and write these to non-volatile storage, so as to use them the next time that it runs, as well as during the current run.
If the ISP can be queried, by the plug-in or the company, this can be done in the fashion of a Web Service. For example, the query might simply be "<askDelimiters/>", where this goes to some port on the ISP's web site. The ISP's reply might be
<delimiters>
<s><messageBody></s>
<e></messageBody></e>
</delimiters>
Consider an ISP with a given user interface. It wants to change this, and also change to new delimiters. It should convey this information to the company prior to the change, and also to any plug-ins that query it directly. In the information that it conveys, it can include a time, after which the new user interface and delimiters take effect. But there are difficulties in standardizing on a uniform time across a network. Different computers can be set to different times, under the control of their owners. The plug-in has no control of the time on its computer. Hence, to handle the transition to new delimiters, the plug-in should have logic to accommodate both for some period of time.
The Aggregator might also hold such delimiter information. And the ISP might send new information to it, if the ISP will be changing its delimiters.
The ISP can also make the information about a future set of delimiters available as a Web Service. Essentially, for a pair of delimiters, there might be an extra set of tags, like <time>8Aug2007</time>. Or it could be written as an attribute, like <delimiter time- '8Aug2007" />. When queried, the ISP's Web Service might return the currently active delimiters, and possibly a future pair.
An ISP might have different user interfaces at any given time. So that its users could choose between these, for example. In general, each interface would have different delimiters. This information can also be communicated to the company or plug-in, in the above fashion. If the ISP decides to make these delimiters available via a Web Service, it could be via the obvious generalization of our example above for a Web Service that offers up one set of delimiters.
It is also possible that different ISPs might standardize on a common user interface. At least to the extent that they use the same delimiters for the message body. They could still customize the visual appearance.
The support of an ISP by the plug-in may involve the ISP furnishing a fee to the company that wrote the plug-in.
3.2 Extracting A Message Body 5
Given that we know the start and end tags of the message body, the plug-in can straightforwardly extract it. Note that the plug-in must start from the first (earliest) start tag, and go to the last end tag, and take everything in between as the message body. This is to prevent a phisher inserting extra start and end tags, to throw off where we extract our links 10 from.
For example, a phisher might cause this message to appear -
15. <!— start of example — >
<messageBody> <!-- ISP writes this -->
Some text and links go here <!-- phisher writes this -->
</messageBody> <!-- phisher ADDS THIS ->
More text goes here <!-- phisher writes this --> 0 A link to the phisher's address <!-- phisher writes this -->
</messageBody> <!-- ISP writes this -->
<!— end of example — >
25 The phisher writes a spurious </messageBody> to try to throw off our plug- in, if it naively stops at the first </messageBody> it meets. So it will not detect the crucial link to the phisher, that follows that delimiter. Hence, we need to parse to the last instance of the end tag.
30 Along these lines, several optional heuristics can be recorded. We call these "styles". For
example, if the starting delimiter is present in the message body, we might set a style flag, called startFound, for example. Likewise, if the ending delimiter is present in the message body, we might set a style flag, called endFound. Also, if the starting delimiter is present in the message body, and we detect a link that is present in the message body, between this delimiter and the true starting delimiter, then we might set a style flag, called linkBeforeFakeStart, for example. Likewise, if we see a link in the body, between an ending delimiter in the body, and the true ending delimiter, then we might set a style flag, called linkAfterFakeEnd, for example.
These optional styles can be used for various means. For example, higher level analysis might take these as indications of what techniques phishers might possibly be using.
3,3 Message Header
It should be appreciated from the above that similar statements can be made for obtaining delimiters for the message header. The plug-in can then use these delimiters to extract the message headers from the web page.
One utility is to let the plug-in attempt to extract the purported sender line. Which could then be used in further validating the message.
Caution needs to be used here, in attempting further analysis of such a header. In many ISPs' display of messages, there is a fundamental difference between the way a header is shown and the way the body is shown. For the latter, few, if any, ISPs add material. Because users expect to see an original body of information. But in the header, an ISP might add links, for functionality that is a convenience to the user. For example, a link next to the sender line might let the user add that sender to her address book. So parsing the header for links may not be as useful as in doing this for the body.
3.4 ISP Puts Links in Message Body
While we have suggested that most users may not want to see the body of a received message altered by the ISP, this may change. So suppose, for whatever reason, an ISP wants to explicitly put in one or more links in a message. These links might be to itself or to third parties (possibly as ads). Our methods here and in the earlier Sections can be easily modified to handle this. The plug-in or the company that wrote the plug-in can query the ISP for a Partner List. This list refers to the base domains that will appear in new links in the message body, that the ISP will write. In general, it will be different from those domains in links in the web page that are outside the display of the message body.
Of course, the ISP can make this list available via a Web Service.
Note that what we consider here is different from an ISP adding links (perhaps to itself) to a user's message, when that is an outgoing message. This is reasonably common amongst some ISPs, as advertising.
3.5 ISP Changes Links in Message Body
An ISP might also modify existing links in a message body. For example, it might change an existing link in the body to point to an address run by the ISP. The rest of the contents of this altered link contain the original link information, modified in some fashion. If the user picks the link, the browser goes to the ISP's address, passing it the arguments in this altered link. The ISP's server at that address uses these arguments to contact the original link.
Currently [2004], at least one major ISP does this, Hotmail, which is owned by Microsoft Corp.
We draw a distinction between these changes and the ISP introducing a brand new link into
the message body.
The plug-in should be able to undo the ISP's changes, in order to extract the original base domain. This treatment is similar to how the plug-in finds the delimiters for the message body. Firstly, there is a hard-wired approach. When the plug-in finds the base domain of the page, and sees that it is a known ISP, it can look up in its internal data if that ISP does this link changing. If so, the plug-in might have a prerecorded list of the ISP's domains and it can check the domain in the link against this list. If there is a match then the plug-in knows that the original domain is in the arguments of the link. Again, for a given ISP, the plug-in may have logic that can unravel these arguments and find the original domain. The computational effort of this to the plug-in is not expected to be high, because the ISP itself has to perform this unraveling on its machine, as quickly as possible, in order to scale to large workloads.
An alternative is for the ISP to make this information available to a query from the plug-in or the company that wrote the plug-in. Or the ISP could send the information to the company. The ISP could implement the query answering as a Web Service.
The information from the ISP is of two types. Firstly, a list of the addresses of its servers, that it writes as the addresses in the changed links. Secondly, it needs to convey information about its encoding scheme for moving the original link into the arguments of the changed link. There may need to be an a priori agreement between the ISP and the plug-in's company as to a simple, standardized language or convention that expresses this mapping. It would then also be of use to other ISPs that wished to change links. Because by describing their mappings in this language, they could still carry out their changes, and their customers could still use our methods to validate their messages. Note that this does not constrain an ISP in what rewriting changes it wishes to make to a link, provided that those changes can be expressed in that language.
Of course, for any information the ISP offers, it could also offer advance information about
any changes it might make.
3.6 Language Independent
It can be seen from the above that the methods are independent of any human language. While it is possible that an ISP's delimiter might have meaning in a given language, as a word or pictogram, its use in the context of the delimiter is separate from that meaning.
3.7 Summary
ISPs have an incentive to conform to our methods described here. And to inform an Aggregator or the plug-in's company in advance of any changes to their user interfaces. This does not constrain their freedom of action in designing unique user interfaces.
The plug-in can then dynamically accommodate their changes, and offer their end users protection against fraudulent electronic messaging that is received by the ISPs.
Section 4. Antivirus Method
4.1 General Data Validation
The ideas in the preceding sections can be simply extended into an ability to validate files or data. There, we showed how our methods treat messages and websites equivalently. Here, using the ideas of a Partner List, Behavior List, Aggregator and the Notphish tag, we will show how to make the following correspondence (the terminology is explained below) -
plug-in < > Loader/Runner browser < > Operating System message/web page < > foreign data
Imagine a user, Sarah, loading a program ("Gamma") onto her computer, where the program claims to be from a reputable company Chi, with a website chi.com. The program might have been downloaded by Sarah from a different website. (Imagine a bug patch for an existing program, where the latter is known to be from Chi, and the patch is promulgated by many websites, for example.) Or, Sarah might have gotten a DVD with this program from some source, i.e. by physical distribution. The program might be in some scripting language, or, in general, in a binary format that can be run by her computer.
One method of verification could be via insertion of a tag into Gamma. That is, if Gamma really came from Chi, then Chi would have placed this tag into it. Preferably at or near the front of Gamma, so that it can be quickly detected, and the contents of this tag read by some kind of loading program ("Loader"). While the format of the tag is broadly arbitrary, we can take a preferred implementation to be the Notphish tag, perhaps in the form of this example -
<notphish a="chi.com" id="1258d" />.
The Loader is a program in the operating system. Here, part of its duties can be similar to those of the browser plug-in discussed earlier. Where the Loader's relationship to the operating system is like the plug-in with respect to the browser. The Loader can read the tag and ask the Aggregator for data about chi.com. First, is it a customer of the Aggregator? If not, then the tag is false, and the Loader might mark the program as invalid (a virus perhaps?) and deal with it accordingly. (Deletion?)
So suppose chi.com is a customer of the Aggregator. The Loader can download from the
Aggregator information associated with the id 1258d. For example, the information might include a hash of the program and a code designating which hash algorithm was used. So the Loader can use this algorithm against Gamma and compare the resultant hash with the
downloaded hash. If different, then Gamma can be marked as suspect.
Also, instead of, or in addition to the above, Chi might at some earlier time have uploaded to the Aggregator a Partner List and Behavioral List for Gamma. Here, the Partner List is a
5. list of network addresses and protocols that Gamma will or might use during its run. In essence, this is the same as the Partner List of earlier sections. There, the plug-in could parse a plain text or HTML message or web page to find its links and their domains and protocols, and compare against the official Partner List. Here, we can envisage the Loader downloading the Partner List and passing it to another program ("Runner") that runs 0 Gamma. Of course, Loader and Runner might be the same program in some imp lementati ons .
Prior to Runner be activated on Gamma, Loader can also attempt to extract network addresses and protocols from Gamma. If any are not on the Partner List, then Loader can 5 mark Gamma as suspect.
Runner can scrutinize any network communications made by Gamma, prior to these connections actually being made. If it finds an address or protocol combination not in the Partner List, then it can terminate Gamma and inform Sarah. This runtime assessment 0 differs from the HTML/plain text message or web page, because those usually have static addresses, which permit an extraction by the plug-in, before the browser displays the data. But for an arbitrary program Gamma, which is not malware, it may have valid reason to generate a dynamic address.
5 For the Partner List, there is an important special case. When it is empty. That is, Gamma is never meant to make a network connection. This gives the Loader and Runner a very strong test on Gamma.
It is also possible that for some Gamma, its Partner List might inherently be any address 0 on the network. Imagine the case where Gamma explicitly lets Sarah give it any address,
for example. Here, the Partner List can have some notation to designate this, so that the Loader and Runner do not need to test any network connections made by Gamma. (Though they may do so based possibly on other, external logic.)
Aside from the Partner List, the Behavior List shows the types of operations that Gamma will or might perform. For example, will Gamma change any existing files? Will it delete any existing files? Will it use any encryption routines already present on the computer? Does it contain encryption routines? Can it run with the privileges of a typical user, or does it need system privileges? Etc.
Our method complements and enhances recent advances in antivirus detection. Traditional methods often involved finding a hash signature of a binary that was manually determined to be a virus. Then, these signatures were combined into a table that the antivirus vendor would periodically download to its customers. But virus writers responded by crafting polymorphic viruses, with each virus instantiation having a unique hash. (Cf. the texts cited in the References at the beginning of this Invention.)
To counteract this, others invented methods that try to discern behavior of an unknown binary. Often, these methods require intensive computations on an unknown program. In essence, they try to determine "bad" files, and to find invariant structures in those files. Then, a suspect file might be searched for the presence of those structures. To some extent, this is necessarily somewhat subjective and usually very computationally intensive. Plus, this requires the antivirus vendor to have very skilled technical personnel who can manually isolate and study new viruses. And during this zero-day time, the customers are vulnerable, before the vendor can issue new code to detect those viruses.
Whereas steps in our method can be run first on Gamma. It is an objective analysis, against definitive "good" files and their properties. If a file has a tag and passes this analysis, then the system knows objectively that the file is okay and can omit the usual conventional behavioral analysis. If the file has a tag and fails our method, then the
system knows objectively that the file is bad, and can also omit the usual behavioral analysis.
But what about a file with a tag, where some behaviors or the Partner List can only be measured by running it in the Runner? Then, before doing this, the operating system might choose to run the usual behavioral analysis. Or, as discussed earlier about the Runner, it can abort Gamma if it deviates from the downloaded Partner List or Behavior List.
It can be seen that the use of an Aggregator with a Partner List and Behavior List against a program is a simple extension of our earlier methods. We consistently deprecate the use of a blacklist or the possibly difficult finding of (subjective) bad properties. Instead, we offer a quicker and objective finding of known good properties.
There is another reason why our method against a file with a tag is objective, compared to a customary subjective approach. The latter is done against a file with an unknown author, in general. Each individual behavior found in the file might indeed be suspect. But occasionally, a real program from a reputable source might also exhibit such behavior. However, the analysis has no access to such information. Hence, of necessity, the analysis must deal in generalities.
In contrast, our method lets a reputable company release a program with several generically suspect behaviors, without it being "hit" by conventional analysis. Each company defines the correct behaviors of its programs, independently of other companies.
Our method improves the efficacy of current antivirus methods by permitting those to focus their attention on unknown files that lack Notphish tags.
4.2 Message Attachments
Our method can also be applied against attachments to messages. An attachment might be some arbitrary file, or an encoding of that file. The latter might arise if the file has binary data, whereupon a common encoding method like Base64 or uuencode could be used to convert the file to an ASCII representation for transmission via email. Typically, many email handling programs require or prefer that messages be restricted to ASCII symbols.
As above, the Notphish tag might be inside the attachment body. Alternatively, the information that would be inside the Notphish tag might be written in the attachment header. In this case, the information might, optionally but preferably, be written in the form "X-Notphish ..." where the "X-" is used to start a line in the attachment header. There might be several such Notphish lines. This "X-" prefixing is the standard method in email for third parties to attach custom lines to a header.
4.3 Patch Updates
For files, as contrasted to messages or web sites, there is one simple extension of a Partner List which is useful for patch updating. Imagine that Gamma from Chi needs to be updated at some point. Chi can furnish to the Aggregator a list of approved sites that have updates to Gamma. Call this perhaps an Updater List, where it is associated with Gamma. It may be that Chi's web site's CPU or bandwidth will be overloaded if many end users go to it directly for Gamma patches. Also, this surge in demand might only be transitory, till most users have gotten the patches. So it is not cost effective for Chi to ramp up with more CPU or bandwidth if that will normally be mostly idle.
Each item in the Updater List might be a network address (like a URL), such that the act of selecting it initiates a download.
Also, when Chi writes the Notphish tag into Gamma, it can put an field into the tag, that says that an Updater List is available on the Aggregator.
Currently, when a program's patches are being disseminated, each company tends to do its own procedures. So if Sarah has several programs on her machine to be updated, she might have to go through different steps for each. Note that this is different from the format that the patches might be in. Those formats are currently mostly standardized. So updates for linux machines might have patches in rpm format, for example.
Here, our method is not concerned about the format of the updates. But about a simple and safe means for finding these on the web. Then, the Loader (or perhaps the Runner) might download the Updater List from the Aggregator, and contact one of those sites for an update.
Above, we discussed how Chi might benefit if its updates could be downloaded from various sites. But the Loader using the Updater List also lets Sarah (or her sysadmin) benefit. It offers the chance of automating these update (and install) steps. Presently, a sysadmin often has to go to each vendor's website, then manually download and install the patches. Many of these steps are specific to each vendor. But the Updater List and the Aggregator act like a standardized interface. And once this exists, automation steps can easily arise.
4.4 Directory
Above, we showed how Sarah's computer can ask the Aggregator about a file with a Notphish tag. This can be extended to the case where she has a directory, d, which has been unloaded into her computer from some external source. In d, there might be a file, call it "Notphish.txt", say. It contains a tag, which might be in the form given above. Here, the value of the id field, when uploaded to the Aggregator, tells it to download information about d. This information might list each file name in d, and recursively in any subdirectories. With each file name is affiliated information like the hash of the file [perhaps]. And possibly a Partner List and Behavior List. If d contains a lot of files, then
the Aggregator might download only a subset of information. And there would be some query possibly by the Loader (or Runner), to ask for the remaining information.
Or, the tag information might be encoded in the name of the Notphish file itself, e.g. "Notphish-bank0.com.txt", with the file being empty.
The Loader can then do a listing of the actual files in d and down into the subdirectories, and, at the simplest level, compare these names with those from the Aggregator. If there are extra files or missing files in d, then the Loader might indicate this to Sarah. The Loader could also compute hashes of some or all of the files, and compare against the correct hash values given by the Aggregator.
For hashing of files, there may be occasion when a file might be meant to have variable content, and hence a hash is not meaningful. The Aggregator reply can indicate this if needed. Or, sometimes a file takes on one of a (small) range of values. So the file might have a set of hashes affiliated with it. Again, the Aggregator reply can indicate this if needed.
4.5 Copyright Protection
Our method can also be used to prevent unintentional copyright violation. Suppose Gamma is not some virus, but a presumed valid instance of a program copyrighted by Chi. If Chi has uploaded to the Aggregator the hash of Gamma (and any other related data), then Sarah's Loader can compare the computed hash to this. If it differs, then Gamma might be an unauthorized modification of the real Gamma from Chi. (This assumes that Gamma is not a polymorphic program, and so can usefully be identified by a hash.)
4.6 Other Uses
It might be asked, if we could identify a program by its hash, then why a need for the
Aggregator to have a Partner List or Behavior List for the program? Indeed, if all the instances of a program can be uniquely identified by the same hash, then this might be true. But there may be cases where each program instance is unique. Perhaps there is a personalization or unique code embedded in the program? The latter code might be some kind of copyright protection. Functionally, it means that Chi has made a polymorphic program, for reasons totally different from those of virus writers. In this case, Chi might choose to parameterize Gamma by its Partner List and Behavior List.
But there is another reason why even if all instances of a program are the same, Chi might still want to upload the program's Lists to the Aggregator. Hitherto, Sarah's computer has only used the downloaded Behavior List to see if Gamma deviates from this list. However, Sarah might have a policy against running even programs whose provenance has been validated, if they have an undesired behavior. Currently, there appears to be no such common description of program behavior documented by the program's author, and disseminated in a validated way, and in a structured data format
(e.g. XML) that can be applied by its users. This action by the author is important.
Because the author has access to the source code and can easily and objectively document the program's behavior. One can imagine, as part of the author's programming environment, that she might have a utility program which can extract this behavior from the source code and put it into a form usable by the Aggregator, with need for only minimal manual intervention by her.
Alternative approaches that involve a third party applying a rule base to the program, to try to find its behavior, are subjective, more computationally intensive and more error prone.
Thus, the recipient, Sarah, can have a program runtime security policy. On her computer, or especially if this computer is part of a corporate network at her employer, there might be various hierarchies of such policies. The actual policy that gets applied against Gamma, when she tries to run it with the Loader or Runner, might be some combination
of these existing policies. For example, if Sarah is considered an experienced user by her company, she might have a policy that lets her run a Gamma that can reformat a non-system disk partition.
Plus, each policy might also have specific cases pertaining to different companies that author these tagged programs. Even though all these companies are customers of the Aggregator, and are thus presumed to be reputable. For example, suppose Sarah's computer runs a Microsoft Corp.'s operating system. She might have a policy that is willing to grant more privileges to a program that has been verified to be from Microsoft. While programs from any other computer company will run under more restricted privileges.
In a similar way, a Partner List of a file or a message or web page can also have a policy applied to it. So that if an entry is considered undesirable under that policy, then the file is not run, or the message or web page is marked as "bad" by the plug-in. Recall also that if the network addresses are dynamically generated during the running of the program, then an analysis of the static code may be insufficient to determine these addresses. Hence it is useful for the author to send a Partner List to the Aggregator.
Section 5. Summary
From the use of a Partner List, Behavior List, Aggregator and a Notphish tag, we can detect in a computationally lightweight and objective manner, phishing messages, pharming web sites, and we can validate the authorship of files, where the latter helps in virus detection. From the case studies in the previous sections, we can generalize our invention as follows.
Consider a user with a computer. She obtains some data. Where the data type could be, but is not limited to, email, web page or file. Where the obtaining could be via an electronic network (wired or wireless) or by other means, like physical distribution using
a DVD or tape. And where the obtaining could have been initiated by her (e.g. she goes to a website using a browser), or it could have been initiated by an external event (e.g. she gets an electronic message or someone at a trade show hands her a DVD).
The data usually has assertions that it was originally from some company or author. Or these assertions might arise out of the context in which the data was obtained. For the latter case, imagine, for example, that she got the data from a DVD, that had a label with that company's name on it, as the author or originator of the data. In general, this purported company is different from whoever or whatever directly passed that data to her.
The data has a tag. (Maybe several tags.) This is some information, written in a standard format, that can identify the company and possibly the data. Her computer extracts the tag and presents the information to an Aggregator. If no such company is supported by the Aggregator, or an id of the data is not recognized, then the Aggregator tells the user that the data is invalid.
If the company is supported by the Aggregator, and optionally that the id of the data is recognized, then the Aggregator downloads to the user's computer various metrics about that data. These metrics were earlier sent to the Aggregator by the company. The metrics include, but are not limited to, one or more of a Partner List, Behavior List and hash/hashes.
The user's computer can independently compute one or more of these metrics from its copy of the data. It compares the computed metrics with the downloaded metrics. If any differ, then the data is invalid (or highly suspect).
The Aggregator acts as a trusted source of information for her computer. One of its crucial tasks is to only have customers that it has validated in some fashion as reputable.
5.1 Further Extensions
In the above, the network that the user's computer uses to talk to the Aggregator, need not be the same as the network that was used to get the data. (Assuming that the data was obtained via a network.) For example, imagine Sarah with a cellphone that has Internet capability and Bluetooth capability. She walks by a billboard with a Bluetooth transceiver. It detects that her phone can receive Bluetooth, and it sends her a message via Bluetooth. Offering a discount on some products made by a company, BigStore.com. Her phone can programmatically try to verify if this message really came from BigStore.com, if it has a tag. The phone can contact the Aggregator on the Internet and use our method. So here, our method spans two different types of networks.
The data does not have to be digital. Our examples of electronic message, website and file were inherently digital. But it is also possible that the data exist in an analog form. For example, consider a sheet of paper with written text. Sarah might have a device (like a scanner) that can read this using optical character recognition. The text could include a tag. And, once the text is scanned, it can be hashed. And this hash can be compared with that from an Aggregator, using the tag.
Variations on this example are possible. The tag might be in a watermark or in glyphs embedded in the page. And where we assume that Sarah's device can also read this encoding and extract the tag.
Or the data might be a barcode on an item, and within this barcode is a tag. So that a barcode reader can read the barcode, extract the tag and contact an Aggregator to validate the barcode.
Another variant involves the use of the location of the tag or of the user. Consider the above example of Sarah with a cellphone walking near a billboard with a Bluetooth transceiver. The tag might have some position information about itself (perhaps GPS coordinates), which Sarah's phone could verify. And this information might be uploaded
to the Aggregator. Or the Aggregator might download position information, that could then be compared with that in the tag. Or, possibly, the tag has no position information. Instead, when Sarah contacts the Aggregator, it downloads approved position information to her. She might then compare this to the position of the tagged item (in this case, the billboard) or to her position. If she is within sufficient proximity (which can also be downloaded from the Aggregator) then the data containing the tag is valid, and she can do a course of action. But if she is out of proximity, then the data is invalid, and that action cannot be done.
Another variant goes back to what can be expressed in a tag. It might have a field that indicates to the user that she should upload to the Aggregator, not just the tag data, but extra information that she supplies. Hitherto, the main focus of our invention has been letting the user validate data. But our method can also be used to perhaps validate the user. The Aggregator takes the uploaded data from the tag and from the user, and applies logic to determine what it will download. One possible use is to charge the user for asking the Aggregator. Or to limit the number of queries per unit time.
Another example involves a fax machine. Junk faxes are a problem. One possible answer involves the fax getting a message. It might have a policy that it will only print a message that can be validated. That is, the message has a tag, and the fax machine computes a hash of the message, say. Then, it uses the tag to ask an Aggregator for the appropriate hash. By comparing the computed hash with that from the Aggregator, the machine can tell if the message came from the purported sender.
Another example involves where the item in question has the physical form of an id of some kind. Like a passport or driver's license. Or a smart card or credit card. All of these often have the name of the user, a serial number and sundry data. These may be represented on the card in both analog and digital form. The analog form is the printing of the data on the item's surface or pages. The digital form might be held in a magnetic strip on the item, for example. Imagine that Sarah and her computer ask for this id from a
person, Ralph. He hands over the id and her computer scans in its digital data. The computer might also have other hardware that can analyze the physical form of the id, to try to see if it is a forgery. For the digital data, Sarah wants to verify that. Assume it has a tag, which describes the entity that issued the id. Her computer finds a hash of the digital data. Then, it contacts the Aggregator and uploads the entity's name and the computed hash. The Aggregator replies whether such a hash exists for that entity.
How does this differ from current usages? The communication between Sarah's computer and the Aggregator can be across an unencrypted channel. If the Aggregator is on the Internet, then due to the increasing ubiquitous and cheap Internet access, this can be an economical method for verification. Currently, in lieu of this, the computer might need an encrypted channel that goes to the issuer. Which can be more expensive than our method.
Plus, our Aggregator does not need to store actual sensitive information like credit card numbers and user names. All it stores is a hash of the combined information.
An extension of this method is for the item to have some data represented only in analog form, e.g. written on the surface of the item. Then the scanner might do an analog scan to read and convert this data to digital form, and combine it with the digital data stored on the item, before hashing.
Note also that Sarah might be superfluous here, unlike her computer. That is, an implementation of our method does not need the manual intervention of Sarah. Instead, Ralph might place his item into the scanner. Or, the scanner might be able to read the information from the item when it comes within a certain proximity of the scanner.
The cheaper cost of our system may enable more frequent id verification, and thus reduce the incidence of fraud.