Table of Contents
Acknowledgments.
Introduction.
Chapter 1: Penetration Testing Web Applications.
Chapter 2: Web Applications: Some Basics.
Chapter 3: Discovery.
Chapter 4: Vulnerability Analysis.
Chapter 5: Attack Simulation Techniques and Tools: Web Server.
Chapter 6: Attack Simulation Techniques and Tools: Web Application.
Chapter 7: Attack Simulation Techniques and Tools: Known Exploits.
Chapter 8: Attack Simulation Techniques and Tools: Web Services.
Chapter 9: Documentation and Presentation.
Chapter 10: Remediation.
Chapter 11: Your Lab.
Appendix A: Basic SQL.
Appendix B: Basic LDAP.
Appendix C: XPath and XQuery.
Appendix D: Injection Attack Dictionaries.
Index.
Read a Sample Chapter
Professional Pen Testing for Web Applications
By Andres Andreu John Wiley & Sons
Copyright © 2006 John Wiley & Sons, Ltd
All right reserved. ISBN: 978-0-471-78966-6
Chapter One
Penetration Testing Web Applications
At the end of the day, it all comes down to code. There are few information security issues out there that cannot be traced back to bad code, lazy coding, ignorant programming, something having to do with bad software, or bad practices in the creation of software. The fact that it all comes down to code is one of the deeper points to pick up about application and software security because programmers hold those keys. Whenever a security engineer sets up a firewall, she is running someone else's code. Whenever a network engineer configures a router, she runs code someone else wrote. Whenever a network security professional runs a port scanner or an automated vulnerability assessment tool, she runs code someone else wrote. Flaws can exist in those sets of code that can go undetected until someone knowledgeable in that area takes the time to audit them (or they stumble upon the discovery). Finding flaws in applications and software is especially challenging with compiled code that is closed in nature. On the other side, even if code is open sourced it sometimestakes deep programming knowledge and experience to read some source code, understand it, extend it, modify it, and secure it.
Security Industry Weaknesses
Since the proliferation of the Internet, Web applications (referred to simply as apps in this book)-from simple personal and corporate informational web sites to complex applications rich in functionality-have been popping up at an astonishing rate. In the business world, it seems as if any entity without a solid Internet presence is looked on as amateurish. However, although these apps might all share a common presence on the Internet, they are all created differently, on different stacks of technology, and exposed to the world in radically different ways.
Vulnerabilities arise from many sources, from the pressures caused by unrealistic development deadlines and limited consideration for security in the application specification typically delivered to developers to the unsuitability of network-level security to provide adequate protection. The following sections introduce each of these problems, which are explored further throughout the book.
Application Development Considerations
These apps are typically written with a focus on functionality, but minimal security. This, unfortunately, is the nature of programming and development, although programmers and developers are amazing at creating great functionality against some finite set of requirements. And typically, as long as those setting forth the requirements for the apps are happy, the programmers and developers have done their job. Security will either be added in later or dealt with on a network level. Historically, this approach to application development is pervasive and has created tons of software full of security holes.
The bottom line is that it is difficult to code as quickly as our jobs require while maintaining security as an area of focus. Creating secure code may mean that the crazed deadlines we are all given cannot be met, and that is a problem-none of us like to miss deadlines. Now I am not stating that developers and programmers are irresponsible-I am sure that if we were all given realistic time frames to code we would all do the right thing in reference to security. But business rather than IT or engineering teams typically drives software deadlines. Ultimately we all pay the price for the rushing and cutting corners.
Limitations of Edge Security Models
Applications are typically deployed behind some thick edge, or network level, security infrastructure (firewalls, Intrusion Detection System (IDS), Intrusion Prevention System (IPS), and so on). Traditionally, they are then left alone, facing the world 24/7 via the public Internet, to do what they have been programmed to do. The problem with this approach is that there always seems to be some level of mystery about the inner workings of these apps for the network, security, and general IT teams that maintain them. These folks most likely have not been forced to get involved at a level of depth that would expose the core of apps and so their mystification is understandable. It also means that they are at a disadvantage when it is time to look under the hood of apps and tell if something is problematic.
The majority of commercially and personally deployed applications write transactional data to some logging mechanism. They traditionally have had very little other inherent security built into them. To make matters worse, it sometimes isn't even the app that is writing data to a logging mechanism, but rather the web server is doing that. The web server is usually a shared resource among numerous apps and sites, which makes these logs that much more difficult to contend with. Reality in the IT industry has proven that most software and security engineering/operations teams do not have the bandwidth to actually review anything that is readily and regularly logged. So the problem isn't only the breach; it is also that internal resources don't realize their apps have been breached. These situations tend to get investigated in a reactive manner.
Breaches are taking place constantly-you can get an idea of the extent by visiting SecurityTracker (http://www.securitytracker.com/archives/category/4.html) and Zone-H (http://www.zone-h.org/en/defacements).
This typical scenario brings to light some of the deficiencies of a security model based strictly on the edge. Unfortunately edge- or network-level security does very little in terms of actually protecting applications. Yet most modern-day IT teams and operations are blissfully comfortable with edge security supposedly providing protection to their data and functional resources. The reality of the matter is that most edge-level security mechanisms identify some web-based attacks as legitimate traffic and they happily watch the traffic flow on by.
The Case for Pen Testing
The false sense of security that exists within the IT industry has spawned the much-needed profession of Web application security and penetration (pen) testing. The focus of this book is web-based application penetration testing and how the results of this can lead to a layered approach of enhanced core-level security coupled with specialized edge security for Web applications. The benefits of being a programmer doing pen testing are due to the deeper levels of app understanding gained through this practice. But other IT professionals will get eye-opening information and education as well.
Industry Preparedness
Application pen testing is a critical discipline within a sound overall IT strategy. But there is a serious problem: the shortage of those with the necessary skill set to properly pen test software, N-tier, and distributed applications. Experience has shown that the typical security specialist or engineer simply lacks the depth of application and software knowledge necessary to be entirely effective when performing these types of audits. Knowledge of code is absolutely necessary to do this effectively.
The present state of affairs with Web applications, and corporate software on the whole, is one of quasimystery. There is a disturbing gap in the industry between the programming community (which focuses on solid functionality) and the security community (which focuses on protection at the network level and policy). The gap exists because, while programmers and web developers are traditionally focused on the functionality of apps running critical processes, and network and security professionals are traditionally edge and possibly even host specialists, no one is looking for security holes in the application code. The mystery, then, is who properly secures the Web apps? The experience of programmers has been that security is not a priority in their application development workloads.
Many businesses do not have in-house application skill sets or resources. Although they have apps that are, and have been, running their business in the background-and from a business perspective everything functions as expected-these apps have typically been outsourced or off-shored for development. This means that there is very little application knowledge on staff. Because of that, one of the things regularly encountered out in the field is very old and unpatched versions of software. This is a huge problem because the most up-to-date software out there typically has enhancements and fixes built into it. But many entities will not always keep up with the latest and greatest software due to a lack of in-house knowledge and experience.
A further consequence of having applications that no one understands is that the folks in-house that are held responsible for these applications refuse to touch them for fear of breaking them. Things that are seemingly simple, like applying server patches, could conceivably wreak havoc on an application. If they do apply the patch, library dependencies can end up broken and the app could start spitting out nasty errors. Anyone who witnesses a fiasco like this makes a clear mental note not to repeat that mistake, which leads to an unwritten no-touch policy.
Sometimes it is not even a human process that causes a problem. If a server runs long enough without being turned off, there is no guarantee it will come back up after a shutdown. The average professional in the IT industry will not want to be the individual that powered that server down. They don't want to deal with the repercussions if the shutdown causes an app to stop working. So there is a distinct preference not to touch anything that is perceived as not broken irrespective of the associated risk.
Many edge-level techniques and tactics leave great areas of risk exposed. For example, the functionality of IDS systems is impressive, but what they watch is dictated by what they are taught. However, the critical question remains, who is doing the teaching? Do they properly understand what they are looking for? Assuming they do, someone or something then has to make sense of the massive amounts of data these systems typically capture. It is an intensive, time-consuming process that in the real world has proven to be a "nice to have," yet the true real-world security value is arguable. IPS systems come with their own set of challenges and weaknesses. The point is that when it comes to Web apps, there are weaknesses, and risk is generally present.
IDS (http://en.wikipedia.org/wiki/Intrusion_Detection) and IPS (http://en.wikipedia.org/wiki/Intrusion-prevention_system) systems are network-level devices that aim to enhance an overall security posture.
Awareness in this area is growing, though. Evidence of this can be seen in movements like these:
Open Web Application Security Project (OWASP)-OWASP (http://www.owasp.org) is dedicated to helping build secure Web applications.
Web Application Security Consortium (WASC)-WASC (http://www.webappsec.org) is an international community of experts focused on best-practices and standards within the Web application security space.
One emerging, very interesting area is that of Web Application Firewalls (WAF). These are devices or software entities that focus entirely on the proper protection of Web applications. These WAF solutions are intended to fill the gap I spoke of earlier. They are capable of properly preventing attacks that edge/network-level firewalls and IDS/IPS systems can't. Yet they operate on the edge and the app's source code may not get touched. You can get information at http://www.modsecurity.org and http://www.cgisecurity.com/questions/webappfirewall.shtml.
As we all know, not all products are created equal, and this is especially so in software. Ivan Ristic and the WASC are going to the great length of formalizing the evaluation criteria of these types of solutions so that anyone can at least have a baseline understanding about the effectiveness of a given solution. As stated on their web site: "The goal of this project is to develop a detailed Web application firewall evaluation criteria; a testing methodology that can be used by any reasonably skilled technician to independently assess the quality of a WAF solution." They have a strong Web app security presence within the consortium, and the criteria seem solid. You can learn more at http://www.webappsec.org/ projects/waf_evaluation/.
Finding the Right Mix of Experience and Methodology
Because the security industry is predominately staffed with network professionals who have migrated to security via firewall, IDS, IPS, work, and so on, it is easy to find a great many misconceptions in the way application pen testing is described and practiced. For starters, there is sometimes an unfortunate misconception that pen testing doesn't follow any disciplined methodology. The thinking is that there is a strong dependence on the tester's experience, and therefore there is a direct correlation between that experience and the difficulty of the specific target. However, even with a good store of relevant experience, without a defined methodology it is easy to make mistakes, generate inaccurate results, waste time, waste money, and finally lose the client's (when servicing external clients and not auditing your own shop) confidence that they will receive an excellent end-result product.
Now, some entities performing this type of work operate with no methodology and this is certainly not a good practice. It is almost as bad as using a methodology that is far too general, or pen testing based on the instinct or knowledge of specific individuals. A common case of this sort of flawed approach to pen testing can be seen in methodologies that preach information gathering, penetration, and documentation. Unfortunately this methodology is pervasive. Far too many companies and IT personnel have the erroneous idea that a penetration test constitutes nothing more than running a security scanner and getting a nicely formatted report with colorful charts at the end of the run. The main fault with this model is that the results depend only on how many problems were discovered, which depends on what the scanner has been taught by its programmers, and that depends on the experience and knowledge base of those particular programmers and possibly some analysts associated with the project. To view one of those reports as complete and comprehensive is a grave oversight to anyone familiar with the way security scanners and other automated tools work and who truly know about the false-positives and false-negatives they produce.
There are countless examples of sloppy and inaccurate pen tests done by big and small consulting companies that lack the right skill set, depend on automated tools exclusively, or both. The reports they provide are based on the superficial results of automated tools without any deeper analysis. This is downright irresponsible and potentially leaves a trusting client needlessly exposed because the tool(s) used, and the people using them, may have missed something.
The Role of Social Engineering
There is great value in coupling Web app pen tests with other social engineering efforts, such as shoulder surfing. Even though this book exclusively focuses on the technical aspects of Web applications, the value of a successful social engineering campaign cannot be negated. Many times the app security experts are fed information from the social engineering efforts of others on a Tiger Team (http://en.wikipedia.org/wiki/Tiger_team). It is a flow of information that could certainly speed up the whole process; just don't rely on it because without it you still have to get results. It is also your responsibility to feed back to the rest of the team any data you discover that may be relevant.
The Bottom Line
There is much more to application pen testing than blindly running a few tools and producing a report. It is imperative that organizations make themselves aware of their risk level in the arena of web technologies. Acting on the awareness is not only critical but now is becoming a legally-based demand. Web-based vulnerabilities and the potential attacks and exploits are growing at alarming rates and they require attention today. The business-related consequences for any organization doing business on the public Internet who fails to take application and data security seriously could be devastating, especially considering the repercussions of non-compliance within areas such as Sarbanes-Oxley.
(Continues...)
Excerpted from Professional Pen Testing for Web Applications by Andres Andreu Copyright © 2006 by John Wiley & Sons, Ltd. Excerpted by permission.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.
loading...