captain holly java blog

Evaluating WebScarab

Posted in security, spring, Uncategorized by mcgyver5 on July 29, 2009

I was asked to do a security assessment on a co-worker’s Cold Fusion application. It is protected on every page by a NOT findnocase(cgi.http_host,cgi.http_referer) check to ensure the request came from the same domain. This is a good way to prevent forced browsing and most url injection attacks because if you mess with the URL, this tag knows it and stops all the shenanigans.
This is where a proxy comes in. I’ve worked a bunch with Paros and some with Burp, but my employer does not allow me to download these without some extra paperwork. Webscarab, for some reason, is allowed. Webscarab is written entirely in Java, has a zippy UI and has widening adoption.

Webscarab allowed me to do forced browsing on the application and learn that the application relied solely on that domain check to make sure the user was authenticated (That is, they could only get to the site through the login form). Webscarab also allowed me to find many XSS bugs.

Webscarab is infinitely scriptable (with beanshell).

Webscarab has a tool that evaluates session identifiers for their strength. I would guess that most web frameworks these days have very strong session identifiers. In fact, I challenge anyone to find an example of a weak session identifier on any web app that shouldn’t be replaced anyway for one hundred other reasons.

Startup Options
Webscarab starts in Lite mode, which is just the web proxy, by default. To get the full meal, you have to start with java -DWebscarab.lite=false -jar webscarab.jar
Default memory is 64MB and this can get used up quickly. Online examples show webscarab having ~510 MB available. This is achieved by adding -Xms32m -Xmx510m to the java startup args. Just like with some other java desktop apps (Like IntelliJ Idea) you can click on the Green|Yellow|Red bar along the bottom of the window to force garbage collection and free up some memory.

Things That Could Be Improved:

  1. Inconsistency: Some features are available through a right click, some through a double click, some from a menu item and others from buttons or tabs somewhere on the screen. Some fields look editable but aren’t. Some are editable on one click, others on two. Some edit fields select the whole field when clicked, but typing appends to the end of the existing entry.
  2. Other screens have a delete button. Not the Proxy Listener Tab. To delete a listener you must stop it. If I a listener fails to start, it may not be stopped and so cannot be deleted. I have to stop any other service using the same port as my listener, THEN start my listener, and THEN stop my listener to delete it
  3. The interface for getting rid of conversations is difficult to use. Webscarab can fill up pretty fast with banal conversations and the only easy way to get rid of them all is a restart. There is a Tools –> remove conversations menu item, but no regex that I enter seems to get rid of conversations.
  4. There should be some way to construct the proxy filters based on existing requests. By this I mean when a request is trapped that you never want to see again, you can flag it in some way to add it to the ignore list.
  5. Judging from several posts to the mailing list, Webscarab only works with Sun’s brand of java.

To address user experience as well as other issues, Webscarab is undergoing a total rewrite. This is currently known as Webscarab NG. They will be using the Spring Rich Client Platform. The new product also has database integration. This is a work in progress and needs lots of testing. So, if you are looking for an open source project to help, this would be an excellent choice. According to the email list, the Webscarab NG project leader has been directing his work at the OWASP Proxy lately. Even though Webscarab NG is in development, development also continues on the current Webscarab.

DRPL: definitely broken

Posted in Uncategorized by mcgyver5 on July 28, 2009

The Denied and Restricted Parties List (DRPL) is kind of a No-Fly list for export restrictions and since Sun has some encryption related technology, it is a national security concern that someone might take the SCJP exam.

After initially being informed that my request to take the exam was denied, today I got an email from SUN saying that I’m not, after all, someone who might do bad stuff.
here is some background email that was attached to my email:

The following individual, as a result of screening, has been identified as being as a potentially non-compliant export customer:

Search Key: US2121923
First and Last Name: Tim McGuire

City: St. Paul

Country: US

Result of DRPL check: Detected

Date and time of denial: Mon Jul 27 10:10:15 MDT 2009

Course Order Numbers: No Numbers Generated.

Reason for denial: Not Available from service.

From here and here I see that a bunch of people have been inconvenienced and aggravated to varying degrees because some mouth breather hasn’t figured out that no-fly lists are a fake idea. Imagine if Macy’s did a surprise sniff test on every 100th customer in their underwear department? This is just like that.

Slowloris vs tomcat

Posted in security, tomcat, Uncategorized by mcgyver5 on June 19, 2009

RSnake has been thinking about a denial of service attack against web servers that involves sending partial http packets to use up number of allowed clients. Sending carefully crafted partial packet causes the server to take A LONG TIME to work on the response to your request, using up its resources and becoming temporarily unavailable to other visitors. Apache HTTPD is mentioned as a server that is vulnerable. IIS is mentioned as one that is not. RSnake, being a realist and not an anti-microsoft evangelist, often says things that make the open source advocates uncomfortable. (“PHP is the bane of my existence” and “Whenever I assess a dot net application I know right off the bat that I’m going to find half the number of vulnerabilities”).

A few notes about Slowloris: It can’t effectively dos a box from windows because it works by creating hundreds of Sockets and Windows only allows a max of 130. It doesn’t crash anything, so it is a gentle tool(haha) It just happens to make web applications unavailable for as long as the attacker wishes. It does, by the way, send out hundreds of packets so it is detectable by the administrator.

To use Slowloris, first establish a timeout for the web server you are attacking:
./ -dns http://localhost -port 8080 -test

this should return some numbers to use for a timeout.

They don’t mention tomcat, so I spent most of the afternoon setting up a machine to see if this tool can DOS tomcat.

drum roll please….

slowloris test

slowloris test

clear that we’ll be using a 5 second timeout for TCP and a 30000 millisecond timeout for http.
./ -dns localhost -port 8080 -timeout 30000 -num 500 -tcpto 5

the above opens 500 sockets and uses a tcp timeout of 5 seconds and looks like this:

slowloris execution

slowloris execution

now, try and connect to the benighted tomcat server.
hmmm. works fine. What gives?  I suspect that as this number of connections (500), I am still able to get a connection.  The first visit takes a really long time, but once I get through, I can use the site normally.  This matches the statement in the documentation that ”    “.  If I raise the number of connections….  It still takes a very long time to load the first page, but thereafter is just as easy to access the application.

When I run slowloris on the same server, however, tomcat is completely DOS-ED.    I’m impressed with the absolute unavailablity of tomcat in relation to the low level of traffic that slowloris generates.

ooooh.   I thought I was supposed to convert 30 seconds into milliseconds.  wrong!  setting the timeout this high  (30,000 seconds) is clearly too high.  When I set it down to 30, slowloris CRUSHED tomcat.  remotely or locally.  As you can see below, setting the timeout correctly allowed many more packets to be sent.

slowloris success

slowloris success

Tagged with: ,

How do you like jadclipse?

Posted in Uncategorized by mcgyver5 on June 5, 2009

I installed jadclipse to see if it was a way to view source of all the libraries I use but don’t have the source code for.

Installation is simple, with the small extra step of downloading jad and telling eclipse where to find the jad.exe. (Window –> Preferences –> Java –> Decompilers –> Jad)

All the jad command line options are represented in the jad dialog.

inner classes and static blocks were decompiled just fine. I did notice that jad left out any braces in if-then blocks that weren’t needed. Nitpicking a bit here, but I can’t read code without braces. The braces were put in when I checked the “show redundant braces” options, but not until I restarted eclipse.

Finally, some classes seem to have a stuck setting causing them to open in class viewer. For instance, Struts Action gets decompiled just fine, but Struts DispatchAction just shows itself in that awful class viewer. Restarting Eclipse made this problem go away too.

Big thumbs up for this very useful eclipse plugin.

thumbs up

thumbs up

How does Sigar work?

Posted in native, Uncategorized by mcgyver5 on June 4, 2009

Sigar is an impressive suite of system monitoring and reporting tools made by a company called Hyperic (which was purchased by SpringSource in May). Hyperic also makes the enterprise Hyperic HQ monitoring tool suite.

Sigar’s basic function is to make underlying system information available to Java. The pieces that come with Sigar include:

  • A set of system info commands like ps, netstat, ifconfig, and df
  • The Sigar object. The Sigar object maintains platform specific internal state. Looking at the source code, we see that this object is passed around all over the place so each part of Sigar knows which platform it is dealing with.
  • The Sigar Proxy is an interface that is implemented by the sigar object which provides caching at the java level.
  • The sigar shell, which is a command shell used to issue sigar commands and queries
  • Sigar.jar library which can be included in java programs
  • hooks into jmx
  • A pager
  • hooks into the windows registry and other windows specific stuff
  • Process Table Query Language. Sigar has its own process table query language (PTQL) that is used to drill into process info and find out things including process children and viewing command line arguments used for a given process. It identifies processes by other attributes besides process ID so that sigar’s identification of the process can persist over time even if the process ID changes via a restart.
  • Hooks into vmware information. Allows java to monitor and control vmware appliances.

Many Sigar features have “toString()” overridden to provide a formatted output similar to what you would expect to see from your system as well as getter methods for you to customize the reporting.

A good example of what Sigar provides is the Top command

The command uses ptql
the simplest usage would be java -jar sigar.jar Top Pid.Pid.eq=3872
which gives a result of:

C:\Downloads\java\hyperic-sigar-1.6.0\hyperic-sigar-1.6.0\sigar-bin\lib>java -jar sigar.jar Top Pid.Pid.eq=3976
←[2J  3:14 PM  up 1 day, 6:50, (load average unknown)
62 processes: 0 sleeping, 62 running, 0 zombie, 0 stopped... 728 threads
CPU states: 0.0% user, 0.0% system, 0.0% nice, 0.0% wait, 100.0% idle
Mem: 2086444K av, 1408404K used, 678040K free
Swap: 4027084K av, 2015628K used, 2011456K free

3976    McGuiT1 Jun1    296M     31M    -       R       0:45    0.0%    C:\Program Files\Microsoft Office\OFFICE11\WINWORD.EX

your java program can use it to get the top resource hogs on the underlying system with something like

String qs = "";
ProcessQuery query;
        try {
            query = this.qf.getQuery(qs);
        } catch (MalformedQueryException e) {
            traceln("parse error: " + qs);
            throw e;
        try {
            long[] pids = query.find(sigar);

Some of the source code contains funny comments:

//XXX currently weak sauce. should end up like netstat command.

I guess “weak sauce” refers to things like


where HEADER is an array of strings.

I found that Sigar’s Netstat command sometimes runs significantly slower than the equivalent system command.
output of Sigar’s netstat command (Loopelapsed time in milliseconds):

Loopelapsed: 4502
[tcp, localhost:5152, localhost:2857, CLOSE_WAIT, ]
Loopelapsed: 1
[tcp, localhost:2658, localhost:2659, ESTABLISHED, ]
Loopelapsed: 0
[tcp, localhost:2659, localhost:2658, ESTABLISHED, ]
Loopelapsed: 0
[tcp, CM03802-5RR31D1:3575,, ESTABLISHED, ]
Loopelapsed: 4503
[tcp, CM03802-5RR31D1:2523,, ESTABLISHED, ]
Loopelapsed: 4502
[tcp, CM03802-5RR31D1:1108,, ESTABLISHED, ]

what in god’s name is it doing? The reason it takes so long is is that after it uses a native method to get a list of connections, it is using the getHostName() method from java’s InetAddress class to try and find a host name connected to the IP address. According to the InetAddress api, this is tied to my system doing a slow reverse DNS lookup. My windows netstat program obviously has some strong sauce and is not using the same slow DNS reverse lookup. I’m not sure why. This is a mystery for another day.

The grunt work in Sigar is handed off to C code with calls like this:

 public native NetConnection[] getNetConnectionList(int flags)
        throws SigarException;

In this case, this hands off the responsibility of getting a list of connections results from a native implementation in C. On windows this uses the IP helper dll called iphlpapi.dll.

Sigar test run from the command line returns ton of system information and a few errors. Most of these are due to permissions. Running as root makes most of these errors go away.

Experiment with the sigar shell java -jar sigar.jar
>sigar [TAB] gets you all the choices:

>sigar ps [TAB] gets you all the options for ps:
sigar> ps

>sigar ps Mem.[TAB]
gets you more options:
sigar> ps Mem.

It is using the reflection package to look at the options available for each method.

    public static Collection getMethodOpNames(Method method) {
        if (method == null) {
            return SOPS;
        Class rtype = method.getReturnType();
        if ((rtype == Character.TYPE) ||
            (rtype == Double.TYPE) ||
            (rtype == Integer.TYPE) ||
            (rtype == Long.TYPE))
            return NOPS;
        return SOPS;

for finding the actual methods, it uses an anonymous static block:

  static {
        Method[] methods = SigarProxy.class.getMethods();
        for (int i=0; i<methods.length; i++) {
            String name = methods[i].getName();
            if (!name.startsWith(PROC_PREFIX)) {
            Class[] params = methods[i].getParameterTypes();
            if (!((params.length == 1) &&
                  (params[0] == Long.TYPE)))
            METHODS.put(name.substring(PROC_PREFIX.length()), methods[i]);

With a static block, the code is run automatically when the class is loaded.

We can see a drawback to using reflection because the tab completion functionality returns some deprecated methods.

Other Problems:

  1. Depending on how you are logged in, sigar may not have access to some system resources
  2. missing a bunch of javadocs. Plenty of room here for you to chip in and write some docs.
Tagged with: , ,

Concurrency. Where are we and how did we get here?

Posted in concurrency by mcgyver5 on June 1, 2009

Concurrency allows multiple parts a program to execute at the same time. Non-concurrent, program statements run one after another. Concurrency allows them to run in parallel, using system resources more efficiently.

Concurrency introduces indeterminate behavior when computer code executes. Essentially, randomness in outcomes is introduced by the hardware.

Concurrency also introduces situations where resources are requested at the same time by two different threads. If the resources are not managed appropriately, the resources or the threads themselves can become unavailable.

Concurrency is an illusion. At the processor level, only one task at a time can be performed. The ability of the processor to switch between tasks fast enough provides an illusion of concurrency. With multiple processors, this is still true because many more tasks can run than the number of processors. A language is “concurrent” to the extent that it can safely take advantage of this context switching.


As computers got fast enough to support virtual memory and fast context switching, it became possible for multiple users to use the same computer system and have the illusion they were the only ones on the system. An individual user still had to do sequential computing. One computation had to wait for another to finish before it could start.

The next leap in technology allowed one user to use concurrent computing so that one long computation could run at the same time as one or more others that didn’t depend on the outcome of the long computation. For example, printing the first page of a graph while the second page was still being processed would save time. An early language that made this possible was Algo68.

A problem that must be solved by all concurrent systems is how to allow simultaneous computations to access the same resources. This problem was described by The Dining Philosopher’s problem in which a group of philosophers sit down to eat a meal which requires two forks (or, in some versions, chopsticks). Each philosopher has one fork and must wait for another to put down theirs so they can eat. This can result in a stalled, or deadlocked, program if each philosopher picks up their fork at the same time and waits for someone else to put theirs down. Once the deadlock problem is solved, the program must be sure that none of the philosphers starves to death. This means that although the program never locks up, it may happen that one of the philosophers never gets a chance to eat.

Various conventions have been used in languages to implement concurrency

  • Shared Resource Annotation. Annotations abstracted the gory details of concurency so that they remained hidden from the programmer. A notation simply identified a resource as shared when first instantiated. Early implementations, however, did not ensure that the same resource could be declared somewhere else and not identified as shared.
  • Messaging. This would be analogous to the philosophers simply talking to one another. In the Erlang language, for example, concurrent processes never share the same memory. They can, however monitor each other and communicate.
  • Monitors are objects that govern resources that might be accessed by concurrent computations. Monitors ensure that only one thread at a time can operate on the resource they govern. They also establish a queue that governs which thread has next access to the resource. In some languages, the resources themselves can be made into monitors. In others, monitors area seperate object that controls access to a resource. In 1974, C.A.R Hoare wrote a famous paper outlining the use of monitors in a
    concurrent operating system. Read the paper here.

mile posts along the way to concurrency:

  1. Assembly. Concurrency was known as “multiprogramming” and was implemented in Assembly. People went insane trying to maintain large concurrent systems written in assembly.
  2. Semaphores. semaphores are flags indicating “yes” the resource is in use and “no” it is free. The “occupied” sign on a port-a-potty is a binary semaphore. PL/1 was the first language to use binary semaphores to protect “critical regions”.
  3. The THE operating system was a multi tasking operating system written in Assembly, but providing ALGOL as the sole programming language available. The ALGOL language “abstracted” the ugly details of concurrency from the programmer, who implemented concurrency through notations, starting the concepts of A) hiding implementation details from people who did not need to see them, and B), protecting resources by using a higher level language.
  4. Simula was the first language to group like resources into things called classes. This allowed a paradigm you will recognize from Java in which you can define protection levels for groupings of resources like private, public, default, protected. The problem with classes in Simula is that outside modules could access the variables of the classes themselves, bypassing the procedures meant to protect the resources. This is possible in Java only by doing something really crazy like using introspection.
  5. Concurrent Pascal In 1974, A man named Hansen developed Concurrent Pascal as the first generally used concurrent programing language. It was the first to use monitors.
    Concurrent Pascal implemented a set of features deemed “required ” by a true concurrent language. These are:

    • The compiler should make sure that all resources that can be accessed by more than one thread be protected by monitors
    • The language should be a high level language in which memory addresses, interrupts, and so on are hidden from the developer. So, things like assembly or C are out. It is possible in those languages to perhaps write a monitor, but it is also possible for some joker to write a program that invades the memory space of the resources you are trying to protect, as there is no way to enforce access control like there is in java with private, protected, and public modifiers
    • The language needs to be modular. When things are modularized, the entire module can be designated as shared or synchronized. In Concurrent Pascal, unlike Simula, variables could be protected. This was vital to concurrency as competing code segments cannot now access a single variable somewhere, (like reaching in a window to adjust a thermostat), they must walk in the front door with permission. A language called Simula had the concept of classes, which were groupings of resources.

    Monitors are a central part of Concurrent Pascal. A monitor is essentially a shared class with explicit queues. A shared class is simply one that has all of its fields private and all of its methods synchronized.

    Explicit Queue: In java, a “monitor” has one queue. When the state of the monitored object changes, all waiting threads are woken up and notified of the change. In a “true” monitor, we could have one queue for each possible state of the watched object so that threads waiting on one aspect of the object to change could all be in the same queue and wouldn’t get notified unless something of interest to them happened.

    Errors with shared resources or time-dependent operations must be detected during compilation. No amount of testing can tell for sure that a shared resource will or will not lock up / finish in a finite time. Thus, the compiler is used to detect whether shared resources are defined as such and that concurrency errors are handled.

  6. Mesa was a high level language developed by Xerox PARC in the late 1970s with strong concurrency and modularity. It was mainly used within Xerox as an experimental and teaching language. One of the requirements in Mesa was to be able to hatch new processes on the fly when they were needed. In earlier concurrent languages such as Algo68, the necessary processes had to be requested and rationed out in the declarations section of the program.
  7. Modula-2+ was a concurrent language that was used to develop operating systems at DEC, most notably the VAX.
  8. Erlang. Erlang was developed in the 1980s by the Erickson Corporation to run their phone switches. It was required to be massively concurrent and scalable. It was open sourced by Erickson and picked up by others including Erickson’s competitor Nortal and some web 2.0 applications such as Facebook chat. It uses the “Actor” model to achieve concurrency. Actors are lightweight processes. They wink in and out all the time like quantum particles and they chat with one another constantly and asynchronously. Reading about actors reminds me very much of web services.

    Actors never share state and thus never need to compete for locks for access to shared data. Instead, actors share data by sending messages that are immutable. Immutable data cannot be modified, so reads do not require a lock.


  9. Java. With monitors established in academic circles as THE way to solve concurrency problems, Java came along with “Shared-state” concurrency and without a “true” monitor. With Shared state, concurrent computations live in the same memory space and it is up to the programmer to make them play nicely with one another.

    This all made P. Brinch Hansen write:

    Java’s most serious mistake was the decision to use the sequential part of the language to
    implement the run-time support for the parallel features.
    In 1975, Concurrent Pascal demonstrated that platform-independent parallel programs (even
    small operating systems) can be written as a secure programming language with monitors. It
    is astounding to me that Java’s insecure parallelism is taken seriously by the programming
    language community a quarter of a century after the invention of monitors and Concurrent
    Pascal. It has no merit.


    With java 5, Java has revamped their concurrency. It introduced a Monitor, deprecated thread methods like suspend() and stop() that enabled unsafe operations by one thread on other threads and could lead to deadlock, and now they have a Concurrency framework. It is still a “shared-state” solution, but with more tools to help the programmer make the threads play nicely together.
    It doesn’t quite meet the classic definition of proper concurrency because the compiler cannot detect errors, but it implements most other things. Here is a full discussion of the concessions java had to make with regards to concurrency and why they argue on Wikipedia that Java isn’t a truly concurrent language.
    straight from sun:

    Monitors provide greater security from system breakdown than Java does.
    Compilers for languages that support monitors can assure that all fields that can be accessed by more than one thread are protected by monitors. They can also assure mutual exclusion to those fields. They can check for the possibility of deadlock due to cycles in the graph of monitor calls. The implementation of these compilers can guarantee that a thread waiting for a condition will gain access to the monitor immediately, so the condition
    will still be true.

  10. Scala. Concurrency is now getting even more important because it is needed to take advantage of multi-core systems (systems with lots of CPU chips instead of just one). It may be that Java’s leap in concurrency is not enough or is too difficult to harness completely to take advantage of multi-core systems.

    So, bringing us up to the current day, Scala was developed by Martin Odersky to run on the Java Virtual Machine. Odersky also developed Generic Java, which was incorporated into Java 1.5. Among other “improvements”, Scala implements concurrency using the “actor” method as opposed to Java’s “shared-state”.

Tagged with: , ,

TCJUG = Spring 3.0

Posted in spring, tcjug by mcgyver5 on May 19, 2009

I went to the May Java Users Group lecture last week.  I came away with the feeling that the Spring Framework was about the coolest thing in the world.  The presentation, however, was mostly about Spring 2.5, with a few new Spring 3.0 things.

The highly anticipated 3.0 feature is the REST support in Spring.  While browsers only implement GET and POST, the new REST wiring in Spring allows you to do such things as tell a form to submit using the idempotent PUT method.  It also allows you to vary your request mapping based on the HTTP method.  For example, every web developer in history has built a form that must both be filled with default data (i.e.  a GET request) and submitted (i.e. a POST request).  The flow ends up at the same URL, but now you can branch behavior (either update the database or select from the database) based on the HTTP method used to reach the form page.

The powerful annotations of 2.5 are enhanced in 3.0, allowing less reliance on XML. The wiring is now defined in the classes themselves. This has the disadvantage of requiring a compile when you change the code, whereas XML did not, but it has the advantage of reducing the number of places you need to change things. It also, as the creator of Spring pointed out in a podcast[1], changes your definition from where and when to just when because the where is implicitly defined by where in the source code you place your annotation.

@requestParam in method signature eliminates the need to pull values out of Request Object.

  • Previously you may have done:   method(HttpServletRequest request, HttpServletResponse response){ String theName= request.getParameter(“theName”);
  • Now, you can do:  method(@RequestParam(“theName”) String theName);

paths for request mapping can have wildcard/variables as in artist/{artistId}/album/{albumId}/

Binding complex types to a form. Instead of just text in a form field, you can inform a simple html text element that it is, in fact, a horse, or a currency or a date, or even part of a larger object like a book order.

flexible method signatures. This is a 2.5 feature. For example, older versions required a return type, here you can go ahead and return void instead of null if your method does not need to return anything.

ETag caching.  An Etag is a header in the response object.  It looks like this:

ETag: 333565bb8c284d8afad71192f209435d

That long string represents a message digest of the content, generated by the server, and serving as a unique identifier. When the browser requests the page again, it sends along this long string and if it matches what the server has stored, the server returns a 304 message, “the resource has not changed”.  This article shows how an implementation might be hand-coded in Spring. 3.0M1 has a filter built in called ShallowEtagHeaderFilter that does what the article suggests.  By shallow they mean, “the generated jsp is the same and even though we worked to generate it, we won’t bother sending it over the wire”.  By deep, they mean that somehow, the underlying data structure can be checked for changes, and the jsp won’t even be generated.  A deep Etag filter is in the “maybe” stage.

Paths in Spring MVC can be altered to, for example, put .json at the end to return json output.

Right now, to validate a form, spring users generally pass the request through a class that implements Validator, which is fine, but once you are high on annotations, you will want to use annotations for validation as well.  This is not yet implemented in Spring 3.0 but it will soon follow <a href=”″>JSR 303 Bean Validation</a> recommendations.

See also:

  1. Java Posse podcast interview with Spring founder Rod Johnson covers some of the same things.
  2. Google Guice and SpringSource have partnered to standardize on annotations
  3. summary of spring 2.5 features.