tag:blogger.com,1999:blog-45317163313366790542024-03-15T09:17:04.614+01:00Andy's Software Engineering CornerThoughts on Java, Language Design, Database- and Web Technologies...andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.comBlogger29125tag:blogger.com,1999:blog-4531716331336679054.post-27673336547250758822020-10-26T12:37:00.000+01:002020-10-26T12:37:13.500+01:00Some weekend fun: Simple Rust, complex maths and having fun in the terminal.. <p>This time a short post, as most of its contents live on GitHub:</p><p><a href="https://gist.github.com/andyHa/485f474a63568e610869ceb966ba9b6a">GitHub GIST: magic.rs</a></p>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com6tag:blogger.com,1999:blog-4531716331336679054.post-57529937235414719582016-08-29T08:11:00.001+02:002016-08-29T08:11:35.103+02:00OSX El Capitan hangs or freezes during boot or after login acutallyI twice had the case that after upgrading and after an update, my MacBook Pro froze doring / after the login screen.<br />
<br />
What didn't help - but what's always worth a try:<br />
<ul>
<li>Deleting the NVRAM (Cmd + P + R + Pwr)</li>
<li>Booting in safe mode (shift + Pwr)</li>
</ul>
So what was the Problem? Kernel Extensions! How to fix it?<br />
<ul>
<li>Boot into recovery mode (Cmd + R+ Pwr)</li>
<li><b>If you have FileVault enabled, open the Disk Utilities, select your main drive (which is grayed out) and select "Unlock..." in the Menu</b></li>
<li>Start a terminal</li>
<li>Remount everything writeable: <code class="plain plain">mount -rw /</code></li>
<li>Naviagte to your Kernel Extensions folder:<code class="plain plain"> cd /Volumes/System/Library/Extensions</code></li>
<li>Look for new or non-standard extensions and move them in a subdirectory called "Unsupported:<code class="plain plain"> mkdir Unsupported; mv Stuff.kext Unsupported/</code></li>
<li><code class="plain plain">The list of standard extensions can be found here: ls /Library/Extensions</code></li>
<li>Good candidates in my case were Logitech Drivers and or a USB to Serial driver<code class="plain plain"></code></li>
<li>Reboot</li>
</ul>
An excellent writeup of all this can be found here: https://www.justinsilver.com/technology/os-x-el-capitan-10-11-1-hanging-on-boot-fixed/<br />Note however, that in this tutorial, the step of unlocking FileVault is missing, which is essential when using an encrypted harddisk.andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-88473494702587316742016-04-11T20:16:00.002+02:002016-04-11T20:21:22.658+02:00Using the Raspberry Pi to program a Microchip PIC (PIC24) device via ICSP<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7Gdw-MmVcMX7UJfl6Sw-Rifrq4NwIEYHrh6E-mb4EEcNP33vCtwuD0KFLZnVxwcqBZjBQNkMhDgClKz9BF0GRmB3eyQQ1iGLThhymg5lrMXLdDcVzd7APZwgp0mugnzXtRf1aCSwEobFG/s1600/Bildschirmfoto+2016-04-11+um+17.25.22.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7Gdw-MmVcMX7UJfl6Sw-Rifrq4NwIEYHrh6E-mb4EEcNP33vCtwuD0KFLZnVxwcqBZjBQNkMhDgClKz9BF0GRmB3eyQQ1iGLThhymg5lrMXLdDcVzd7APZwgp0mugnzXtRf1aCSwEobFG/s320/Bildschirmfoto+2016-04-11+um+17.25.22.png" width="320" /></a>Why would someone want to do that? Well, basically for two reasons. First, the classic: "this should be possible" - so lets try it, spend the better part of three weekends and some nights to finally get it working. Might sound stupid, but to me, this is still the best way to really learn a technology or technical topic. Starting with basic C++ skills, I learned a lot about PIC assembler (yep, ICSP for PICs is basically sending a bunch of op codes to the device), PIC memory layout and of course PIC ICSP and controlling GPIOs of the Raspberry Pi.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOD4RLsTPhfPwODJCKt6MW7ubmYdZt1y9703Glb8uXRJ5H1bPL09bzFlYLNVFu9iTSjtDdRzhhrlqD6rui9fjeqM0fPG13Q8ZnKjfBfweFQcrF5i05XOPsizEdkOSFw5VI-LzlTmLiSt9W/s1600/2016-02-27+20.49.21.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOD4RLsTPhfPwODJCKt6MW7ubmYdZt1y9703Glb8uXRJ5H1bPL09bzFlYLNVFu9iTSjtDdRzhhrlqD6rui9fjeqM0fPG13Q8ZnKjfBfweFQcrF5i05XOPsizEdkOSFw5VI-LzlTmLiSt9W/s320/2016-02-27+20.49.21.jpg" width="320" /></a>What's the second reason? In my case, I have a PIC connected to the Pi anyway, talking to each other via RS232 - so I figured I could use the spare pins on the connector to assign some GPIOs to the appropriate PINs on the PIC and I don't need a second connector for ICSP, the board is quite full anyway. Also I can update the firmware without having to build a bootloader (which is a topic on its own).<br />
<br />
I case you're curious, the whole project will eventually be a driver board for my 3 axis CNC. Currently I have a PC + Arduino + GBRL-Shield, but I'm not happy with the setup. So I'm planning on building a web based controller on the Pi. This way I can control the machine using a cheap Galaxy Tab A or my mobile Phone. Also I can directly upload the G-Code (produced by Fusion 360 - AWESOME!) from my laptop - either directly or via my NAS or Dropbox or the like (both machines are quite a bit apart).<br />
<br />
<i>OMG Andy, all this has already been built</i>?! Yes - it is still fun to try it yourself and learn a whole lot new :-)<br />
<br />
Because I found a lot of inspiration and help in the projects of others, I tried and document my code <strike>quite well</strike> a bit. Also I added a bit of a description on how it all works - so if you're into ICSP / PICs / GPIOs have a look here: <a href="https://github.com/andyHa/OpenCobra/blob/master/raspicsp/README.md" target="_blank">OpenCobra on GitHub</a>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-37907849023046055262016-02-06T15:22:00.003+01:002016-02-09T08:43:13.891+01:00C code always runs way faster than Java, right? Wrong!So we all know the prejudice that Java being interpreted is slow and that C being compiled and optimized runs very fast. Well as you might know, the picture is quite different.<br />
<br />
<i>TL;DR Java is faster for constellations, where the JIT can perform inlining as all methods/functions are visible whereas the C compiler cannot perform optimizations accross compilation units (think of libraries etc.).</i><br />
<br />
A C compiler takes the C code as input, compiles and optimizes it and generates machine code for a specific CPU or architecture to be executed. This leads to an executable which can be directly run on the given machine without further steps. Java on the other hand, has an intermediate step: Bytecode. So the Java compiler takes Java code as input and generates bytecode, which is basically machine code for an abstract machine. Now for each (popular) CPU architecture there is a Java Virual Machine, which simulates this abstract machine and executes (interprets) the generated bytecode. And this is as slow as it sounds. But on the other hand, bytecode is quite portable, as the same output will run on all platforms - hence the slogan "<i>Write once, run everywhere</i>".<br />
<br />
Now with the approach described above it would be rather "<i>write once, wait everywhere</i>" as the interpreter would be quite slow. So what a modern JVM does is <b>just in time</b> compilation. This means the JVM internally translates the bytecode into machine code for the CPU at hands. But as this process is quite complex, the <b>Hotspot JVM</b> (the one most commonly used) only does this for code fragments which are executed often enough (hence the name <b>Hotspot</b>). Next to being faster at startup (interpreter starts right away, JIT compiler kicks in as needed) this has another benefit: The hotspot JIT known already what part of the code is called frequently and what not - so it might use that while optimizing the output - and this is where our example comes into play.<br />
<br />
Now before having a look at my tiny, totally made up example, let me note, that Java has a lot of features like dynamic dispatching (calling a method on an interface) which also comes with runtime overhead. So Java code is probably easier to write but will still generally be slower than C code. However, when it comes to pure number crunching, like in my example below, there are interesting things to discover.<br />
<br />
So without further talk, here is the example C code:<br />
<h4>
<span style="font-family: "courier new" , "courier" , monospace;">test.c:</span></h4>
<span style="font-family: "courier new" , "courier" , monospace;">int compute(int i);<br /><br />int test(int i);<br /> </span><br />
<span style="font-family: "courier new" , "courier" , monospace;">int main(int argc, char** argv) {<br /> int sum = 0;<br /> for(int l = 0; l < 1000; l++) {<br /> int i = 0;<br /> while(i < 2000000) {<br /> if (test(i))<br /> sum += compute(i);<br /> i++;<br /> } <br /> }<br /> return sum;<br />} </span><br />
<h4>
<span style="font-family: "courier new" , "courier" , monospace;">test1.c:</span></h4>
<span style="font-family: "courier new" , "courier" , monospace;">int compute(int i) {<br /> return i + 1;<br />}<br /><br />int test(int i) {<br /> return i % 3;<br />}</span><br />
<br />
<span style="font-family: inherit;">Now what the main function actually computes isn't important at all. The point is that it calls two functions (<span style="font-family: "courier new" , "courier" , monospace;">test</span> and <span style="font-family: "courier new" , "courier" , monospace;">compute</span>) very often and that those functions are in anther compilation unit (<span style="font-family: "courier new" , "courier" , monospace;">test1.c</span>). Now lets compile and run the program:</span><br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">> gcc -O2 -c test1.c</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">> gcc -O2 -c test.c</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">> gcc test.o test1.o</span><br />
<span style="font-family: inherit;"><span style="font-family: "courier new" , "courier" , monospace;">> time ./a.out<br /><br />real 0m6.693s<br />user 0m6.674s<br />sys 0m0.012s</span></span><br />
<br />
<span style="font-family: inherit;">So this takes about <b>6.6 seconds</b> to perform the computation. Now let's have a look at the Java program:</span><br />
<h4>
<span style="font-family: "courier new" , "courier" , monospace;">Test.java:</span></h4>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"><span style="color: #cc7832;">public class </span>Test {
<span style="color: #cc7832;">private static int </span><span style="color: #ffc66d;">test</span>(<span style="color: #cc7832;">int </span>i) {
<span style="color: #cc7832;">return </span>i % <span style="color: #6897bb;">3</span><span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span>}
<span style="color: #cc7832;">private static int </span><span style="color: #ffc66d;">compute</span>(<span style="color: #cc7832;">int </span>i) {
<span style="color: #cc7832;">return </span>i + <span style="color: #6897bb;">1</span><span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span>}
<span style="color: #cc7832;">private static int </span><span style="color: #ffc66d;">exec</span>() {
<span style="color: #cc7832;">int </span>sum = <span style="color: #6897bb;">0</span><span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span></span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"><span style="color: #cc7832;"> for </span>(<span style="color: #cc7832;">int </span>l = <span style="color: #6897bb;">0</span><span style="color: #cc7832;">; </span>l < <span style="color: #6897bb;">1000</span><span style="color: #cc7832;">; </span>l++) {
<span style="color: #cc7832;">int </span>i = <span style="color: #6897bb;">0</span><span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span></span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"><span style="color: #cc7832;"> while </span>(i < <span style="color: #6897bb;">2000000</span>) {
<span style="color: #cc7832;">if </span>(<span style="font-style: italic;">test</span>(i) != <span style="color: #6897bb;">0</span>) {
sum += <span style="font-style: italic;">compute</span>(i)<span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span></span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"><span style="color: #cc7832;"> </span>}
i++<span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span></span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"><span style="color: #cc7832;"> </span>}
}
<span style="color: #cc7832;">return </span>sum<span style="color: #cc7832;">;</span><span style="color: #cc7832;"> </span></span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"><span style="color: #cc7832;"> </span>}
<span style="color: #cc7832;">public static void </span><span style="color: #ffc66d;">main</span>(String[] args) {
System.out.println(<span style="font-style: italic;">exec</span>()<span style="color: #cc7832;">);</span><span style="color: #cc7832;"> </span> </span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"> }
} </span></pre>
<pre style="color: #a9b7c6; font-family: "Menlo";"><span style="background-color: white;"> </span></pre>
<span style="font-family: inherit;"><span style="font-family: inherit;"><span style="font-family: "courier new" , "courier" , monospace;"><span style="font-family: inherit;">Now lets compile and execute this:</span></span></span></span><br />
<br />
<span style="font-family: inherit;"><span style="font-family: "courier new" , "courier" , monospace;">> javac Test.java</span></span><br />
<span style="font-family: inherit;"><span style="font-family: "courier new" , "courier" , monospace;">> time java Test<br /><br />real 0m3.411s<br />user 0m3.395s<br />sys 0m0.030s</span></span><br />
<br />
<span style="font-family: inherit;">So <span style="font-family: inherit;">taking</span> <b>3.4 seconds</b>, Java is quite faster for this simple task (and this even includes the slow startup of the JVM). The question is why? <span style="font-family: inherit;">And the</span> answer of course is, that the JIT can perform code optimizations that the C compiler can't. In our case it is function inlining. As we defined our two tiny functions in their own compilation unit, the comiler cannot inline those when compiling <span style="font-family: "courier new" , "courier" , monospace;">test.c</span> - on the other hand, the JIT has all methods at hand and can perform aggressive inlining and hence the compiled code is way faster. </span><br />
<br />
<span style="font-family: inherit;">So is that a totally exotic and made-up example which never occurs in real life? Yes and no. Of course it is an extreme case but think about all the libraries you include in your code. All those methods cannot be considered for optimization in C whereas in Java it does not matter from where the byte code comes. As it is all present in the running JVM, the JIT can optimize at its heart content. Of course there is a dirty trick in C to lower this pain: Marcos. This is, in my eyes, one of the mayor reasons, why so many libraries in C still use macros instead of proper functions - with all the problems and headache that comes with them.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Now before the flamewars start: Both of these languages have their strenghs and weaknesses and both have there place in the world of software engineering. This post was only written to open your eyes to the magic and wonders that a modern JVM makes happen each and every day.</span><br />
<span style="font-family: inherit;"><span style="font-family: "courier new" , "courier" , monospace;"><br /></span><br /> </span>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-71956857432893204802015-06-01T22:11:00.001+02:002015-06-01T22:11:19.555+02:00Highlighting Checkstyle Links using Maven and IntelliJ IDEAAlthough <b>IntelliJ IDEA</b> has en excellent <b>Maven</b> integration, it doesn not recognize file references or file links in the output of Maven commands. One such generator of file links is <b>checkstyle</b> which generates an output like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNt3trFYqOhFJdEPFJAIuq7tjNG6s_Fj21IQZVONM1Jk-Sj5i01Ku3JirFA0JfNJgP1iDgzRSPxIuJEyHzVL8aFWrs69LBOoETJTZFALcJKUBu2gGPv1HsUhELUZWE_zujZ3BKP8og_g0w/s1600/Bildschirmfoto+2015-06-01+um+22.04.53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="460" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiNt3trFYqOhFJdEPFJAIuq7tjNG6s_Fj21IQZVONM1Jk-Sj5i01Ku3JirFA0JfNJgP1iDgzRSPxIuJEyHzVL8aFWrs69LBOoETJTZFALcJKUBu2gGPv1HsUhELUZWE_zujZ3BKP8og_g0w/s640/Bildschirmfoto+2015-06-01+um+22.04.53.png" width="640" /></a></div>
<br />
Now our live would be a lot easier, if we could just click on the message to fix the issue. Luckily, with a litte hack, this is possible: IntelliJ provides a possibility to define custom output filters for "<i>External Tools</i>". Therefore navigate to "<i>Preferences > Tools > External Tools</i>". Add a new one with "<i>mvn</i>" as command and "<i>validate</i>" or whatever triggers checkstyle as parameter.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPQdHKsx_RC8T1tNvGzhA0UaDiZI1mRVzZa-knEz80iWeNzMxoSmKRFpHV6a5QFyXOLc1qPt9d2yoTcbGMzG7HMzrG6lMhRHfzfJD7YZjgDSVDt0vswuiVdn8hXWUBLfb7wwehrNIgbcgJ/s1600/Bildschirmfoto+2015-06-01+um+22.04.05.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="418" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPQdHKsx_RC8T1tNvGzhA0UaDiZI1mRVzZa-knEz80iWeNzMxoSmKRFpHV6a5QFyXOLc1qPt9d2yoTcbGMzG7HMzrG6lMhRHfzfJD7YZjgDSVDt0vswuiVdn8hXWUBLfb7wwehrNIgbcgJ/s640/Bildschirmfoto+2015-06-01+um+22.04.05.png" width="640" /></a></div>
<br />
Then click on "<i>Output Filters</i>" and a a Filter with an arbitrary name and "<span style="font-family: "Courier New",Courier,monospace;">$FILE_PATH$:$LINE$(:$COLUMN$)?.*</span>" as Regular Expression.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrzXICepe4Gki4ZqgVRaK2tDSrkbGIaN8qlXU_HSFo_2jcthLqgy6xYfgQBvq51CP4OaeglR7_gu2-ck-Hb-Tq6k45rdYArmZuwvwzGnJrj8lko1tPlYfrOwSpSf0TnCrzfYNOmnkBIwqA/s1600/Bildschirmfoto+2015-06-01+um+22.05.31.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrzXICepe4Gki4ZqgVRaK2tDSrkbGIaN8qlXU_HSFo_2jcthLqgy6xYfgQBvq51CP4OaeglR7_gu2-ck-Hb-Tq6k45rdYArmZuwvwzGnJrj8lko1tPlYfrOwSpSf0TnCrzfYNOmnkBIwqA/s1600/Bildschirmfoto+2015-06-01+um+22.05.31.png" /> </a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
If you now choose "<i>Tools -> External Tools -> Checkstyle</i>" Maven will run again producing a nicely linked output: </div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvJB9EphG8Z0FTgThtBOk0k_VSuC0G9f7sWnsB1uYudXIhwlvsdSiK08529UZkyIPgdoZ-lj9ZcixABTn5urz2gyTqFyUK19g3f9tPdK2niu1PqByDmlu4sJAxxN1E43VO6lUv1zO0_9sw/s1600/Bildschirmfoto+2015-06-01+um+22.05.59.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvJB9EphG8Z0FTgThtBOk0k_VSuC0G9f7sWnsB1uYudXIhwlvsdSiK08529UZkyIPgdoZ-lj9ZcixABTn5urz2gyTqFyUK19g3f9tPdK2niu1PqByDmlu4sJAxxN1E43VO6lUv1zO0_9sw/s640/Bildschirmfoto+2015-06-01+um+22.05.59.png" width="640" /></a><br />
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-82187313676962375262015-05-20T23:13:00.002+02:002015-05-20T23:13:10.398+02:00Fixing Logjam for PoundThe <a href="https://weakdh.org/sysadmin.html" target="_blank">recently discovered problem "Logjam"</a> in TLS (or the Diffie Hellman algorithm to be exact) is also present in <a href="http://www.apsis.ch/pound/" target="_blank">pound</a> by <b>Apsis</b>. Especially if you're using a pre-build binary via <i>apt-get</i> or <i>rpm</i>, as the DH parameters are built into the pound binary itself.<br />
<br />
So, to block the support of DH Export, it is enough to change or specify a "<i>Ciphers</i>" setting:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">ListenHTTPS<br /> ...</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ...<br /> Ciphers "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA"</span><br />
<span style="font-family: "Courier New",Courier,monospace;">End</span><br />
<br />
<br />
However, to be absolutely on the safe side, I'd recommend to compile your <b>own</b> binary with 2048 bit long DH params (the default ones are "just" 1024 bit anyway).<br />
<br />
<br />
<br />
<br />Luckily the steps are quite simple and straight forward:<br />
<ol>
<li><span style="font-family: "Courier New",Courier,monospace;">wget http://www.apsis.ch/pound/Pound-2.7.tgz</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">tar -xzf Pound-2.7.tgz</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">cd Pound-2.7</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">./configure --with-dh=2048 --prefix= --exec_prefix=/usr</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">make</span></li>
<li><span style="font-family: "Courier New",Courier,monospace;">make install</span></li>
</ol>
This will look for pound.cfg in <b>/etc/pound.cfg</b> (Debian uses <i>/etc/pound/pound.cfg</i>) and install pound in <b>/usr/sbin</b> - just like the Debian/Ubuntu package does - so you can keep / reuse their init.d script. <br />
<br />
Also consider adding "<span style="font-family: "Courier New",Courier,monospace;">Disable SSLv3</span>" (just above Ciphers) to disable SSL3 which is considered insecure.<br />
<br />
Using all this will give you a solid <b>A-</b> on <a href="https://www.ssllabs.com/ssltest/analyze.html">https://www.ssllabs.com/ssltest/analyze.html</a>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-25669944605623109472015-04-21T09:18:00.000+02:002019-08-23T16:37:41.725+02:00Using Rhino with Java 8Java brings Nashorn as new JavaScript implementation for <b>JSR 223</b> (javax.scripting). While this is certainly great news (Nashorn is way faster than Rhino by directly generating Java code), it comes with some challenges: <b>Nashorn is not 100% compatible with Rhino</b>.<br />
<br />
Rhino had some extensions and more or less other interpretations on how to combine the Java world with JavaScript. Therefore you cannot simply replace Rhino by Nashorn. One case (which ruined our day) is that you cannot call static methods on instances. Therefore we had to get Rhino up and running in Java 8 until we have our scripts re-written.<br />
<br />
Although there is an extensive documentation available in <a href="https://wiki.openjdk.java.net/display/Nashorn/Using+Rhino+JSR-223+engine+with+JDK8" rel="nofollow" target="_blank">java.net</a>, it is a bit confusing (some URLs are wrong, some steps are missing). So here are the steps which worked for us:<br />
<br />
<ol>
<li>Download Rhino: <a class="external-link" href="https://github.com/downloads/mozilla/rhino/rhino1_7R4.zip" rel="nofollow" target="_blank">https://github.com/downloads/mozilla/rhino/rhino1_7R4.zip</a> </li>
<li>Download JSR-223: svn checkout svn checkout https://svn.java.net/svn/scripting~svn<br /><i>Yes that is a ~ in the URL!</i></li>
<li>cd scripting~svn/trunk/engines/javascript/lib</li>
<li>Copy the js.jar from rhino1_7R4.zip into this directory (replace the existing js.jar)</li>
<li>cd ../make</li>
<li>ant clean all</li>
<li>Copy ../build/js-engine.jar AND js.jar (of Rhino) into your classpath</li>
<li>Now change:<br /><br /><span style="font-family: "courier new" , "courier" , monospace;">ScriptEngineManager manager = new ScriptEngineManager();<br />ScriptEngine engine = manager.getEngineByName("<b>js</b>");</span><br /> <br />to:<br /> <br /><span style="font-family: "courier new" , "courier" , monospace;">ScriptEngineManager manager = new ScriptEngineManager();<br />ScriptEngine engine = manager.getEngineByName("<b>rhino</b>"); </span></li>
</ol>
<br />
<span style="font-family: inherit;">That's all you need to backport Rhino to Java 8. </span><br />
<br />
<span style="font-family: inherit;">Update: Here's another tutorial on this Topic: </span><a data-saferedirecturl="https://www.google.com/url?q=https://www.javacodegeeks.com/java-8-features-tutorial.html&source=gmail&ust=1566657411756000&usg=AFQjCNEhK4HLu1HFUW23otF-rxmleS7uFg" href="https://www.javacodegeeks.com/java-8-features-tutorial.html" style="-webkit-text-stroke-width: 0px; background-color: white; color: #1155cc; font-family: Arial, Helvetica, sans-serif; font-size: small; font-style: normal; font-variant-caps: normal; font-variant-ligatures: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px;" target="_blank">Java 8 Features Tutorial</a>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-84740905708438548252015-02-02T07:42:00.001+01:002015-02-02T07:42:42.136+01:00A better MessageFormat for JavaThe <b><span style="font-family: inherit;">MessageFormat</span></b> class is widely used by Java, especially when it comes to internationalisation. At first sight, using it is simple and straight forward. Define a pattern like "<span style="font-family: "Courier New",Courier,monospace;">There are {0} files on {1}</span>", either in Java - or better in a in a .properties file. Create a new instance of MessageFormat and supply arguments for the two parameters when calling the format method. This create a formatted string like "<span style="font-family: "Courier New",Courier,monospace;">There are 100 files on /dev/sda</span>".<br />
<br />
Couldn't be easier, right? Yep, but there's still room for improvement. One such improvement is replacing parameter indices with names. "<span style="font-family: "Courier New",Courier,monospace;">There are ${numberOfFiles} on ${disk}</span>", provides way more context to the poor soul having to translate a properties files.<br />
<br />
Another improvement are optional sections. Imagine you have to represent a person as string. It can have a salutation, a firstname and a lastname. Depending on what a user entered, the salutation and or the firstname might be empty. Valid combinations could be "<span style="font-family: "Courier New",Courier,monospace;">Mr. John Foo</span>", "<span style="font-family: "Courier New",Courier,monospace;">John Foo</span>", "<span style="font-family: "Courier New",Courier,monospace;">Mr. Foo</span>". Using a pattern like "<span style="font-family: "Courier New",Courier,monospace;">[${salutation} ][${firstname} ]${lastname}</span>" is enough to create this output if optional patterns are supported. That idea is, that blocks in angular brackets are only output if at least one enclosed parameter is replaced with a non-null value.<br />
<br />
All this is implemented by the <a href="https://github.com/scireum/sirius-kernel/blob/master/src/main/java/sirius/kernel/nls/Formatter.java">Formatter</a> class provided by <a href="https://github.com/scireum/sirius-kernel">Sirius</a>. Using it is quite simple. For internationalisation, use <span style="font-family: "Courier New",Courier,monospace;">NLS.fmtr("Property.key").set("paramName", value).format()</span>. Note that Sirius automatically loads all properties files and makes them available using the <a href="https://github.com/scireum/sirius-kernel/blob/master/src/main/java/sirius/kernel/nls/NLS.java">NLS</a> class. To use the smart formatting capabilities a formatter can be directly instantiated like this: <span style="font-family: "Courier New",Courier,monospace;">Formatter.create("${foo}[ ${bar}]").set("foo", foo).set("bar", bar).smartFormat()</span>.<br />
<br />
<br />
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-86378319730182796752014-06-12T09:12:00.000+02:002014-06-12T09:12:44.295+02:00JavaMail can be evil (and force you to restart your app server)JavaMail always had an interesting approach when it comes to its configuration. Basically you have to fill an untyped map or <b>Properties</b> structure and hope for the correct interpretation. Countless tutorials on the net show the minimal properties required to make it work (send / receive mails).<br />
<br />
However, as we painfully just learned, there are some lesser known properties you should probably take care of, which is timeout settings for socket IO. By default, JavaMail uses an <b>infinite</b> timeout for all socket operations (connect, IO, ...)!<br />
<br />
Now suppose you have a cluster of SMTP servers which handle outgoing mail, accessed via a DNS round robin. If one of those servers fail, which happens to be the one JavaMail wanted to connect to, your mail sending thread will hang - forever! This is exactly what happened to us and we needed to perform some real nasty magic to avoid tragedy.<br />
<br />
Therefore, we now set timeouts for all operations:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> String MAIL_SMTP_CONNECTIONTIMEOUT ="mail.smtp.connectiontimeout";<br /> String MAIL_SMTP_TIMEOUT = "mail.smtp.timeout";<br /> String MAIL_SMTP_WRITETIMEOUT = "mail.smtp.writetimeout";</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> String MAIL_SOCKET_TIMEOUT = "60000"; </span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> // Set a fixed timeout of 60s for all operations - </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> // the default timeout is "infinite"<br /> props.put(MAIL_SMTP_CONNECTIONTIMEOUT, MAIL_SOCKET_TIMEOUT);<br /> props.put(MAIL_SMTP_TIMEOUT, MAIL_SOCKET_TIMEOUT);<br /> props.put(MAIL_SMTP_WRITETIMEOUT, MAIL_SOCKET_TIMEOUT);</span><br />
<br />
Also, if you plan to access DNS round robin based services (like amazon S3) or in our case a mail cluster, don't forget to also configure the DNS cache tiemout of Java (which is also <b>infinite</b> by default):<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> // Only cache DNS lookups for 10 seconds </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> java.security.Security.setProperty("networkaddress.cache.ttl","10");</span><br />
<br />
And while we're at it, for us it turned out to be a good idea to set all encodings to UTF-8 (independent of the underlying OS) to provide a stable environment:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> System.setProperty("file.encoding", Charsets.UTF_8.name());<br /> System.setProperty("mail.mime.charset", Charsets.UTF_8.name());</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">...you don't want to care about stuff like this at all? Feel free to use our open source <a href="http://sirius-lib.net/">Java library SIRIUS</a>, which takes care of all that by providing a neat fluet API for sending mails:</span><br />
<span style="font-family: inherit;"><a href="https://github.com/scireum/sirius/blob/develop/web/src/sirius/web/mails/MailService.java">Sources on GitHub</a></span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">An example usage can be found in the <a href="https://github.com/scireum/sirius/blob/develop/web/src/sirius/web/health/Cluster.java#L289">cluster manager</a>:</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> @Part<br /> private MailService ms;<br /><br /> private void alertClusterFailure() {<br /> ...<br /> ms.createEmail()</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> .useMailTemplate("system-alert", ctx)</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> .toEmail(receiver).send();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ...<br /> }</span><br />
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-91799344322855136732014-02-20T10:01:00.001+01:002014-02-20T10:01:21.882+01:00Multithreaded Java - Screencast on the synchronized keyword<b>synchronized</b> is quite well known in the Java community. Due to its early implementation which had a significant runtime overhead, it has quite a bad image. In modern JVMs this isn't the case anymore - still there's something to look out for.<br />
<br />
Watch the screencast to learn more:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/Jo7ikXARgtM?feature=player_embedded' frameborder='0'></iframe></div>
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-86139263122059021322014-02-20T09:56:00.002+01:002014-02-20T09:56:39.705+01:00Multithreaded Java - Screencast on the volatile keyword<b>volatile</b> is probably one of the least known keywords in Java. Still it serves an important purpose - an not knowing about it might ruin your day....<br />
<br />
Watch this screencast to learn more:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/SC2jXxOPe5E?feature=player_embedded' frameborder='0'></iframe></div>
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-457690506097056662014-02-03T22:56:00.002+01:002014-02-03T22:57:12.729+01:00Version Numbering Scheme - Yet another approach<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUyIjdx0UA1gE9PTtsncuhLqyD9JWXpe8-_KNhKM3TgXZmXhc8mnaE0K8NvV2f49oKJG1qkhHERRxOCPsv5qH5ic9O5pHHCLOHdfM0UQho60ctVst9Bx2XMouSMyhgCWnPCgYx1dYW_nfC/s1600/datecode.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgUyIjdx0UA1gE9PTtsncuhLqyD9JWXpe8-_KNhKM3TgXZmXhc8mnaE0K8NvV2f49oKJG1qkhHERRxOCPsv5qH5ic9O5pHHCLOHdfM0UQho60ctVst9Bx2XMouSMyhgCWnPCgYx1dYW_nfC/s1600/datecode.png" /></a></div>
<br />
Version numbering schemes are probably one of the few things we software engineers have more than sort algorithms. However, there's always room for one more.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
While the classic approach of <b>MAJOR.MINOR.PATCH</b> (e.g. <i>1.8.2</i>) works quite well for libraries or products which are distributed in a broad manner, it is still not as easy as it seems. <i>What is a major change</i>? <i>What a minor</i>? <i>What comes after 1.9</i>? <i>2.0</i> or <i>1.10</i>? There are tons of examples where this classic approach fails, Java being one of the most prominent examples. <br />
<br />
One the other hand, this approach is almost perfectly suited for libraries, as the rules are quite obvious here:<br />
<ul>
<li>increment minor version for every release (<b>2.4 -> 2.5</b>)</li>
<li>increment major version when a backward incompatible change was made (<b>2.4 -> 3.0</b>)</li>
<li>increment the patch level for each update, which only fixed bugs but didn't add functionality (<b>2.4 -> 2.4.1</b>) </li>
</ul>
However, for software which runs in the cloud or is only delivered to a number of customers, the distinction is not always this clear. As we do not distinguish between minor or major updates (<i>ask our sales guys, each release is a major step forward</i>), we ended up using the build numbers of our <a href="http://jenkins-ci.org/" target="_blank">Jenkins build server</a> as version number.<br />
<br />
Although this approach works quite well, there are two problems with it:<br />
<ol>
<li>You need a build server which issues consecutive build numbers</li>
<li>Without looking at the build server, you cannot tell the age of a release (<i>How much older is <b>BUILD-51</b> compared to BUILD-<b>52</b>?</i>)</li>
</ol>
Therefore we now started to switch to another approch for our <a href="http://sirius-lib.net/" target="_blank">SIRIUS</a> based products: Inspired by date code placed on ICs, we started to use the same codes for our releases. A date code consists of four digits, the first two being the year and the second two being the week number. So this blog post would have <b>1406</b> as version.<br />
<br />
As we don't perform more than one release per week, a version number is always unique. Furthermore these numbers are quite short and easy to remember (compared to full dates like <i>foo-20130527</i>). Still they provide a rough information concerning the release date.<br />
<br />
<i>Now as I said, this scheme is not superior over others. It's just a good solution for our problem. Use it if you like it, ignore it otherwise ;-) </i>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-1517433581773532252014-01-07T22:36:00.003+01:002014-02-04T13:46:47.075+01:00Making HTTP content compression work in netty 4<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWJf5edV8u2QHO23hmdfFlq3qQyINifYhXXaoK8YxDYG7cgtZmhaphI2x4zVVDGej_BdTSJqLMruyQHP2B40kyVeuu9___-G9CKhKonrB8ZlYEOttgqThhHgvyaiorIE61LH4H9T0CrczD/s1600/apollo.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWJf5edV8u2QHO23hmdfFlq3qQyINifYhXXaoK8YxDYG7cgtZmhaphI2x4zVVDGej_BdTSJqLMruyQHP2B40kyVeuu9___-G9CKhKonrB8ZlYEOttgqThhHgvyaiorIE61LH4H9T0CrczD/s1600/apollo.jpg" /></a></div>
<a href="http://netty.io/" target="_blank">Netty</a> is really a great framework providing all the things needed to build a high performance HTTP server. The nice thing is, that nearly everything comes out of the box and has just to be put together in the right way. And <b>content compression</b> (gzip or deflate) is no exception. However, when it comes to compressing static content I stumbled quite a few times before everything worked as expected:<br />
<br />
<br />
<br />
<br />
<br />
<b>Update:</b> <i>First of all, widely used tools like wget use HTTP 1.0 and not HTTP 1.1 - therefore we cannot always deliver a chunked response (we have to live with disabling compression then). Also note that the netty guys had pretty much the same idea now: <a href="https://github.com/netty/netty/blob/master/codec-http/src/main/java/io/netty/handler/codec/http/HttpChunkedInput.java" target="_blank">HttpChunkedInput</a> - The problem with HTTP 1.0 or non compressable responses (see SmartContentCompressor below) however remains...</i><br />
<br />
Based on the <a href="https://github.com/netty/netty/tree/master/example/src/main/java/io/netty/example/http/file" target="_blank">http/file</a> example provided by netty I used to following approach to serve static files (same as used in netty 3.6.6):<br />
<br />
<div class="line" id="LC174">
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"><span class="n">RandomAccessFile</span> <span class="n">raf</span><span class="o"> =</span> <span class="k">new</span> <span class="n">RandomAccessFile</span><span class="o">(</span><span class="n">file</span><span class="o">,</span> <span class="s">"r"</span><span class="o">);</span></span></span></div>
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"><span class="n">HttpResponse</span> <span class="n">response</span> <span class="o">=</span> <span class="k">new</span> <span class="n">DefaultHttpResponse</span><span class="o">(</span><span class="n">HTTP_1_1</span><span class="o">,</span> <span class="n">OK</span><span class="o">);</span> </span></span><br />
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"><span class="n">ctx</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="n">response</span><span class="o">);</span></span></span><br />
<br />
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"><span class="k">if</span> <span class="o">(</span><span class="n">useSendFile</span><span class="o">)</span> <span class="o">{</span></span></span><br />
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"> <span class="n">ctx</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="k">new</span> <span class="n">DefaultFileRegion</span><span class="o">(</span><span class="n">raf</span><span class="o">.</span><span class="na">getChannel</span><span class="o">(),</span> <span class="mi">0</span><span class="o">,</span> <span class="n">fileLength</span><span class="o">)</span><span class="o">);</span></span></span><br />
<div class="line" id="LC197">
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"><span class="o">}</span> <span class="k">else</span> <span class="o">{</span></span></span></div>
<div class="line" id="LC199">
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"> <span class="n">ctx</span><span class="o">.</span><span class="na">write</span><span class="o">(</span><span class="k">new</span> <span class="n">ChunkedFile</span><span class="o">(</span><span class="n">raf</span><span class="o">,</span> <span class="mi">0</span><span class="o">,</span> <span class="n">fileLength</span><span class="o">,</span> <span class="mi">8192</span><span class="o">)</span><span class="o">);</span></span></span></div>
<div class="line" id="LC200">
<span style="background-color: white;"><span style="font-family: "Courier New",Courier,monospace;"><span class="o">}</span></span></span></div>
<div class="line" id="LC200">
</div>
<div class="line" id="LC200">
<span class="o">However, as soon as I added a <b>HttpContentCompressor</b> to the pipeline, Firefox failed with a message like "invalid content encoding".</span></div>
<div class="line" id="LC200">
</div>
<div class="line" id="LC200">
<span class="o">As it turns out, the <a href="https://github.com/netty/netty/blob/master/codec-http/src/main/java/io/netty/handler/codec/http/HttpContentCompressor.java" target="_blank">HttpContentCompressor</a> expects <a href="https://github.com/netty/netty/blob/master/codec-http/src/main/java/io/netty/handler/codec/http/HttpContent.java" target="_blank">HttpContent</a> objects as input chunks to be compressed. However, the <a href="https://github.com/netty/netty/blob/master/handler/src/main/java/io/netty/handler/stream/ChunkedWriteHandler.java" target="_blank">ChunkedWriteHandler</a> directly sent <b>ByteBufs</b> to the downstream. Also sending a <b>FileRegion</b> (useSendFile=true) left the content compressor unimpressed.</span></div>
<div class="line" id="LC200">
</div>
<div class="line" id="LC200">
<span class="o">In order to overcome this problem I create a class named <a href="https://github.com/scireum/sirius/blob/develop/web/src/sirius/web/http/ChunkedInputAdapter.java" target="_blank">ChunkedInputAdapter</a> which takes a <b>ChunkedInput<ByteBuf></b> and represents <b>ChunkedInput<HttpContent></b>. However, two things still weren't satisfying: First, <b>FileRegions</b> and the <i>zero-copy capbility</i> still couldn't be used and second, already compressed files like <b>JPEGs</b> will be compressed again. Therefore I sublassed <b>HttpContentCompressor</b> with a class called <a href="https://github.com/scireum/sirius/blob/develop/web/src/sirius/web/http/SmartHttpContentCompressor.java" target="_blank">SmartContentCompressor</a>. This class check if either a header "<i>Content-Encoding: Identity</i>" or a <i>specific content-type</i> or a <i>content-length less than 1 kB</i> is present. In there cases the content compression is bypassed.</span></div>
<div class="line" id="LC200">
<span class="o"><br /></span></div>
<div class="line" id="LC200">
<span class="o">Using this combination permits to use both, content compression when it is useful and the zero copy capability if the file is already compressed.</span></div>
<div class="line" id="LC200">
</div>
<div class="line" id="LC200">
<span class="o">All the sources mentioned above are open sourced under the MIT license and part of the <a href="http://sirius-lib.net/" target="_blank">S</a><a href="https://www.blogger.com/null" target="_blank">IRIUS framework</a>. </span></div>
<div class="line" id="LC201">
<br /></div>
andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-89801961969491125412013-12-25T22:52:00.000+01:002013-12-26T16:25:04.561+01:00Getting a simple CDC demo (serial port via USB) working with a PIC32 (PIC32MX575) This is just a quick write-up to save others some debugging time. The adventure is called: Get the "cdc_serial_emulator" example supplied of <b>Microchip Harmony</b> (/harmony/v0_70b/apps/usb/device/cdc_serial_emulator) up and running on my custom PIC32 board.<br />
<br />
Once one understands the inner workings of the firmeware, the first steps are quite simple. First of all I modified the code which assumed we would run on an Explorer Board. Therefore I commented everything uncompilable and unneeded in <i>bsp_sys_init.c</i> out. <br />
<br />
Now that I was able to compile the project, I was ready to enter the USB descriptor hell. And believe it or not, my Windows 7 machine was the most helpful tool for my first steps. Every time I plugged my board in, those annoying "USB device connected" sounds let me know if the communication worked or not. Of course, it didn't by default. The reason is the clock frequency of the PIC. The USB module has an internal PLL which generates the required 48 MHz. This PLL is fed by the main oscillator and therefore everything needs to be setup correctly. I used (for no special reason) a 20 MHz crystal instead of the 8 MHz crystal assumed by the example.<br />
<br />
Therefore I needed to tweak the config settings found in system_init.c:<i><span style="font-family: "Courier New",Courier,monospace;"> </span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;">OSCO Pin(OSCIOFNC) = Enable<br />Primary Oscillator Configuration(POSCMOD) = External (Highspeed)<br />Secondary Oscillator Enable(FSOSCEN) = Disabled<br />Oscillator Selection Bits(FNOSC) = Primary osc with PLL</span></i><br />
<br />
<b><span style="font-family: "Courier New",Courier,monospace;">#pragma config OSCIOFNC = ON, POSCMOD = HS, FSOSCEN = OFF, FNOSC = PRIPLL </span></b><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">PLL Input Divider (FPLLIDIV) = Divide by 5 </span><br />
<i><span style="font-family: "Courier New",Courier,monospace;"></span></i><i><span style="font-family: "Courier New",Courier,monospace;">(20 Mhz / 5 = 4 MHz) </span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;">PLL Multiplier (FPLLMUL) = Multiply by 20</span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: "Courier New",Courier,monospace;"></span></span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;">(4 MHz * 20 = 80MHz) </span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;">System PLL Output Clock Divider (FPLLODIV) = Divide by 1</span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;"></span></i><i><span style="font-family: "Courier New",Courier,monospace;">(80 MHz / 1 = 80 MHz system clock) </span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;">Peripheral Clock Divisor (FPBDIV) = Divide by 1 </span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;">(80 MHz / 1 = 80 MHz peripheral clock)</span></i><br />
<i><br /><span style="font-family: "Courier New",Courier,monospace;">Watchdog Timer Enable (FWDTEN) = Disabled<br />Clock Switching and Monitor Selection (FCKSM) </span></i><br />
<i><span style="font-family: "Courier New",Courier,monospace;"> = Clock Switch Enable, Fail Safe Clock Monitoring Enable</span></i><br />
<br />
<b><span style="font-family: "Courier New",Courier,monospace;">#pragma config FPLLIDIV = DIV_5, FPLLMUL = MUL_20, FPLLODIV = DIV_1</span></b><br />
<b><span style="font-family: "Courier New",Courier,monospace;">#pragma config FWDTEN = OFF, FCKSM = CSECME, FPBDIV = DIV_1</span></b><span style="font-family: "Courier New",Courier,monospace;"><i> </i></span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"><i>Enable PLL for USB clock generation </i></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;"><b>#pragma config UPLLEN = ON</b><br /><br /><i>Divide external input clock by 5 before it is fed into the USB PLL.</i></span><br />
<i><span style="font-family: "Courier New",Courier,monospace;">20 MHz / 5 = 4 MHz (This is multiplied by 24 and then divided by 2 - see Reference Manual Page 27-3). This will result in the desired 48 MHz USB reference clock.</span></i><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /><b>#pragma config UPLLIDIV = DIV_5</b></span><br />
<br />
As this only took 3 evenings to figure out, this can be considered the easy part...The problem was, that neither Linux nor OSX would recognize the device as serial port (the device itself was found). Therefore it was clear that the timing was ok, but the USB descriptor wasn't.<br />
<br />
After days of debugging (mostly on OSX using USBProbe) I switched to Linux and tried "lsusb -v". And there you go: The CDC-Descriptor provided by microchip was/is wrong! lsusb said something like "INVALID CDC(UNION) 0x04 0x24 0x06 0x00. Looking at the descriptor, this was really the input in <i>system_config.c</i>:<br />
<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> // Size of the descriptor<br /> sizeof(USB_CDC_UNION_FUNCTIONAL_DESCRIPTOR_HEADER),<br /> // CS_INTERFACE<br /> USB_CDC_DESC_CS_INTERFACE,<br /> // Type of functional descriptor<br /> USB_CDC_FUNCTIONAL_UNION,<br /> //com interface number<br /> 0,</span><br />
<br />
<span style="font-family: inherit;">Compared to any other CDC device I tried, this was missing one byte as the record (descriptor) has to look like LENGTH, TYPE, SUBTYPE, MASTER_INTERFACE, SLAVE_INTERFACE - so this was clearly missing one byte. Therefore I changed to code to:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: inherit;"><span style="font-family: "Courier New",Courier,monospace;">0x05, // Size (5 bytes)<br />0x24, // DescriptorType: CS_INTERFACE<br />0x06, // DescriptorSubtype: Union Functional Descriptor<br />0x00, // MasterInterface<br />0x01, // SlaveInterface0</span></span><br />
<br />
Of course, that didn't work out of the box - as I changed the descriptor by adding one byte, I had to fix the size at the begin of the descriptor:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">/* Configuration 1 Descriptor */<br />const uint8_t configDescriptor1[]={<br /> <br /> /* Configuration Descriptor */<br /> //sizeof(USB_CFG_DSC), // Size of this descriptor in bytes<br /> 0x09,<br /> // CONFIGURATION descriptor type<br /> USB_DESCRIPTOR_CONFIGURATION,<br /> // Total length of data for this cfg<br /> <b>67</b>,0, <i>// This was 66 originally</i></span><br />
<br />
Using this, both, my Mac and Linux machines now recognize the device and supply a tty for it :-)andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-76188918617638520362013-12-21T20:38:00.000+01:002014-02-11T13:54:04.602+01:00How to write one of the fastest expression evaluators in Java<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Granted, the title is a bit of an attention grabber, but nevertheless true (You course you never trust a benchmark you didn't fake yourself - but that's another story).<br />
<br />
So last week I was looking for a small and usable library to evaluate mathematical expressions. I almost directly stumbled upon <a href="http://stackoverflow.com/questions/2226863/whats-a-good-library-for-parsing-mathematical-expressions-in-java" target="_blank">this stackoverflow post</a>. The recommended library (<a href="https://github.com/darius/expr" target="_blank">Expr</a>) is really quite fast and had almost everything I needed. However, what it didn't provide was the ability to limit the scope of variables (everything is in one global namespace within the VM).<br />
<br />
Therefore I did, what one normally shouldn't do: I reinvented the wheel and wrote my own parser / evaluator. It was a rainy saturday anyway so I thought a small recursive descending parser, an AST which simplifies and eventually computes expressions along with a little helper for managing variables doesn't seem to be a big deal anyway. And it wasn't. I had an initial implementation up and running quite fast. Once I had some tests giving me confidence that it computed everything the right way, I wanted to know how fast the evaluator was, compared to other libraries mentioned in the original post. Having not hand-optimized every inner loop and everything, I had't much expectations, some of the libraries are commercial ones afterall. So I was quite suprised when I looked at the results. The list below shows a micro benchmark which evaluates the same expression using the respective library. The measurements for <b>parsii</b>, which is my library, were done using the final version, which performs some simplifications, like pre-evaluating constant expressions. However, no "black magic" like bytecode generation or anything in that league is done.<br />
<br />
For a performance measurement the expression "<span style="font-family: "Courier New",Courier,monospace;">2 + (7 - 5) * 3.14159 * x^(12-10) + sin(-3.141)</span>" was evaluated with <b>x</b> running from 0 to 1000000. This was done 10 times to warm the JIT and then 15 times again of which the average execution time was taken:<br />
<ul>
<li><span style="font-family: "Courier New",Courier,monospace;"><b>PARSII</b>: 28.3 ms</span></li>
<span style="font-family: "Courier New",Courier,monospace;">
</span>
<li><span style="font-family: "Courier New",Courier,monospace;"><b>EXPR</b>: 37.2 ms</span></li>
<span style="font-family: "Courier New",Courier,monospace;">
</span>
<li><span style="font-family: "Courier New",Courier,monospace;"><b>MathEval</b>: 7748.5 ms</span></li>
<span style="font-family: "Courier New",Courier,monospace;">
</span>
<li><span style="font-family: "Courier New",Courier,monospace;"><b>JEP</b>: 647.0 ms</span></li>
<span style="font-family: "Courier New",Courier,monospace;">
</span>
<li><span style="font-family: "Courier New",Courier,monospace;"><b>MESP</b>: 220.8 ms</span></li>
<span style="font-family: "Courier New",Courier,monospace;">
</span>
<li><span style="font-family: "Courier New",Courier,monospace;"><b>JFEP</b>: 274.3 ms </span></li>
<span style="font-family: "Courier New",Courier,monospace;">
</span></ul>
Now I'm sure, each of these libraries has their own strengths, so they can't be directly compared. Still it's amazing to see that a simple implementation can compete quite well.<br />
<br />
For those of you who are not too deep into compiler contruction, here's a small outline of how it works:<br />
<br />
As any parser or compiler parsii uses the classic approach of having a <a href="https://github.com/scireum/parsii/blob/master/src/parsii/tokenizer/Tokenizer.java" target="_blank">tokenizer</a>, which converts a stream of characters into a stream of tokens. Therefore "<span style="font-family: "Courier New",Courier,monospace;">4 + 3 *8</span>" which is <span style="font-family: "Courier New",Courier,monospace;">'4', ' ', '+', ' ', '3' , ' ', '*', '8' </span>as character array will be converted into:<br />
<ul>
<li> 4 (INTEGER)</li>
<li>+ (SYMBOL)</li>
<li>3 (INTEGER)</li>
<li><span id="goog_791884122"> * (SYMBOL)</span></li>
<li><span id="goog_791884122">8 (INTEGER) </span><span id="goog_791884123"></span></li>
</ul>
The tokenizer takes a look at the current charater, then decides what kind of token it is looking at and then reads all characters, which belong to that token. Each token has a type, textual contents and knows the position (line and character) where it started. A lot of in-depth tutorials are available on the net, so I won't go into any details here. You can take a look at the source code, but as I said, it is just a basic naiive implementation.<br />
<br />
The <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/Parser.java" target="_blank">parser</a> which translates the given stream of tokens into an AST (Abstract Syntax Tree) which can then be evaluated, is a classic recursive descending parser. This is one of the simplest ways to build a parser, as it is completely written by hand and not generated by a tool. A parser like this basically contains a method for every syntax rule.<br />
<br />
Again a lot of tutrials for this kind of parsers are available. However, what most example leave out is <i>proper error handling.</i> Next to parsing an expression correcly and fast, good error handling is one of the central aspects of a good parser. And it's not that hard: As you can see in the source code, the parser never throws an exception while parsing the expression. All errors are collected and the parser continues to go on as long as possible. Even though after the first error, the resulting AST cannot be evaluated correctly, it is important to go on as we can and should report as many errors as possible in one run. The same approach is used for the tokenizer, as reports malformed tokens, like decimal numbers with two decimal separators, to the same list of errors.<br />
<br />
Evaluating an AST which is the result of a parsed expressions is quite easy. Each node of the syntax tree has an evaluate method which will be called by its parent node, starting from the root node. The result of eval here, is the result of evaluating the expression. A basic example of this approach can be found in <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/BinaryOperation.java" target="_blank">BinaryOperation</a>, which represents operations like +, -, * and so on.<br />
<h4>
In order to improve evaluation time a bit, three optimizations are performed:</h4>
First, after parsing the AST is reduced calling a method called <b>simplify</b> on the root node, which propagates to each child node. Each node then decides if a simpler representation of the own sub-expression can be found. As an example: For <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/BinaryOperation.java#L79" target="_blank">binary operations</a>, we check if both operands are constant (numbers). In that case, we evaluate the expression and returns a new constant containing the result of the operation. The same is done for functions where all parameters are constant.<br />
<br />
The second optimization is done when using variables in expressions. The naiive approach here is to use a map and read or write the values of the variable when needed. While this certainly works, a lot of lookups while be performed. Therefore we have a special class called <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/Variable.java" target="_blank"><b>Variable</b></a> which contains the name and the numeric value of the variable. When an expression is parsed, the variable is <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/Scope.java#L105" target="_blank">looked up once</a> in the scope (which is basically just a map) and then used from now on. As each lookup returns the same instance, variable access when evaluating expressions is as cheap as a field read or write, as we just access the <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/Variable.java#L63" target="_blank"><b>value</b></a> field of <b>Variable</b>.<br />
<br />
The third and last optimization won't probably often come into play. But as it is simple to realize, it was implemented anyway. It basically goes by the name "lazy evaluation" and is used when calling functions. A function does not automatically evaluate all its arguments and then perform the function call iteself. It rather looks at the arguments and can deciede by iteself, which argument to evaluate and which not. An example where this is used, can be found in the <a href="https://github.com/scireum/parsii/blob/master/src/parsii/eval/Functions.java#L277" target="_blank"><b>if</b> function</a>.<br />
<br />
<b>parsii</b> is licensed under the MIT license. All sources can be found on <a href="https://github.com/scireum/parsii" target="_blank">GitHub</a> along with a pre-compiled jar<br />
<br />
<span style="color: #999999;"><span style="font-size: x-small;">(Speedy Gonzales Image-(c) by http://benztownbranding.wordpress.com/2011/11/29/become-speedy-gonzales-or-the-short-cuts-imperium/) </span></span>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-61840809116426613322013-09-19T09:54:00.002+02:002013-09-19T13:23:29.072+02:00How to kill Java with a Regular ExpressionWe recently stumbled upon a phenomen we absolutely weren't aware of: You can kill any Java IDE and also any Java process with a simple regular expression...<br />
<br />
Back in university, I was taught that regular expressions, which are called <a href="http://en.wikipedia.org/wiki/Regular_grammar" target="_blank">regular grammers</a> or <a href="http://en.wikipedia.org/wiki/Chomsky_hierarchy" target="_blank">type 3 grammers</a> always end up in an finite state automaton and can therefore be processed in linear time (input length doubles, processing time doubles). However, that's only true for "sane" expressions. A regular expression can also result in an non-deterministic finite state automaton and things can get messed up quite bad.<br />
<br />
Consider the expression: <span style="font-family: "Courier New",Courier,monospace;">(0*)*A</span> This will any number of zeros, followed by an upper case A. Now if you use <a href="http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#find%28%29" target="_blank">Matcher.find()</a> for this expression, everything is fine as long as there is a match in the input. However, if you call this, with "<span style="font-family: "Courier New",Courier,monospace;">00000000000000000000</span>" as input, your program will hang (and so will the regex console in Eclipse or IntelliJ and every (Java-based) online regex service).<br />
<br />
What at first glance looks like an infinite loop, truns out to be <b>catastrophic backtracking</b>. What this basically means is, that the matcher detects, that no A was found at the end of the input. Now the outer quantifier goes on step back - the inner one forward and again - no result. Therefore the matcher goes back step by step retrying all combinations to find a match. It will eventually return (without a match) but the complexity (and therefore the runtime) of this is expotential (adding one character to the input doubles the runtime). A detailed description can be found here: <a href="http://www.regular-expressions.info/catastrophic.html" target="_blank">catastrophic backtracking</a><br />
<br />
Here are some runtimes I measured (which almost exactly double for each character added):<br />
<span style="font-family: "Courier New",Courier,monospace;"><br />0000000000: 0.1ms<br />00000000000: 0.2ms<br />000000000000: 0.7ms<br />0000000000000: 1.3ms<br />00000000000000: 1.7ms<br />000000000000000: 3.5ms<br />0000000000000000: 7.2ms<br />00000000000000000: 13.9ms<br />000000000000000000: 27.5ms<br />0000000000000000000: 55.5ms<br />00000000000000000000: 113.0ms<br />000000000000000000000: 226.4ms<br />0000000000000000000000: 439.1ms<br />00000000000000000000000: 886.0ms </span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
As a little side-note: For micro benchmarks like this, you always need to "warm" up the JVM as the HotSpot JIT will jump in at some point and optimize the code. Therefore the first run looks like this:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">0000000000: 6.8ms<br />00000000000: 11.8ms<br />000000000000: 25.5ms<br /><b>0000000000000: 39.5ms<br />00000000000000: 6.3ms <- JIT jumped in and started to translate<br />000000000000000: 5.4ms to native code.<br />0000000000000000: 7.1ms</b><br />00000000000000000: 14.2ms<br />000000000000000000: 26.8ms<br />0000000000000000000: 54.4ms<br />00000000000000000000: 109.6ms<br />000000000000000000000: 222.1ms<br />0000000000000000000000: 439.2ms<br />00000000000000000000000: 885.6ms</span><br />
<br />
So what's the take-away here? If you're running a server application or anything critical used by many users, don't let them enter regular expressions unless you really trust them. There are regex implementations out there, which detect this problem and abort, but Java (up to JDK 8) doesn't.<br />
<br />
<b>Note:</b> <i>You can test this with your local IDE or a small Java program to your hearts content - but please don't start to knock out all the regex tester websites out there. Those guys provide a nice tool free of charge, so it would be quite unfair..</i><br />
<br />
Here is the tiny benchmark I used:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">public class Test {<br /> public static void main(String[] args) {<br /> for (int runs = 0; runs < 2; runs++) {<br /> Pattern pattern = Pattern.compile("(0*)*A");<br /> // Run from 5 to 25 characters<br /> for (int length = 5; length < 25; length++) {<br /> // Build input of specified length<br /> String input = "";<br /> for (int i = 0; i < length; i++) { input += "0"; }<br /> <br /> // Measure the average duration of two calls... <br /> long start = System.nanoTime();<br /> for (int i = 0; i < 2; i++) {<br /> pattern.matcher(input).find();<br /> }<br /> System.out.println(input + ": " </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> + ((System.nanoTime() - start) / 2000000d) </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> + "ms");<br /> }<br /> }<br /> }<br />} </span><br />
<br />
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-7428079529955784652013-09-04T13:44:00.000+02:002013-09-04T17:55:07.244+02:00A Monoflop class for easier looping...Whether you want to join the contents of a collection or build a URL query string. There are lot's of cases where you have to handle to first iteraton of a loop a bit different from all others. Often I used this construct:<br />
<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;"><i>List</i><String> listToJoin = ...</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><i>boolean</i> first = true;</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><i>StringBuilder</i> result = new <i>StringBuilder</i>();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><b>for</b>(<i>String</i> item : listToJoin) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if (!first) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> result.append(", ");</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> first = false; </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> result.append(item);</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;"><i>System</i>.out.println(result);</span></blockquote>
Yes, this works like a charm, but just looks plain ugly. And don't ask how long one has to debug if you get the position of the <span style="font-family: "Courier New",Courier,monospace;">first = false</span> wrong.<br />
<br />
A simple class called <span style="font-family: "Courier New",Courier,monospace;">Monoflop</span> can help here. A properly commented version can be found on <a href="https://github.com/scireum/sirius/blob/develop/kernel/src/sirius/kernel/commons/Monoflop.java" target="_blank">GitHub</a>, but a very short version will do here:<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;"><b>public</b> <b>class</b> Monoflop {<br /><br /> <b>private</b> <i>boolean</i> toggled = false;</span></blockquote>
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;"> /**<br /> * Reads and returns the internal state.</span><span style="font-family: "Courier New",Courier,monospace;"><br /> * Toggles it to true once.<br /> */<br /> <b>public</b> <i>boolean</i> successiveCall() {<br /> <b>if</b> (toggled) {<br /> return <i>true</i>;<br /> }<br /> toggled = <i>true</i>;<br /> <b>return</b> <i>false</i>;<br /> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><span style="font-family: "Courier New",Courier,monospace;"> /**<br /> * Inverse of successiveCall</span><span style="font-family: "Courier New",Courier,monospace;"><br /> */</span> <b><br /> public</b> <i>boolean</i> firstCall() {<br /> <b>return</b> !successiveCall();<br /> }<br /><br />}</span></blockquote>
Using this neat helper results in the following:<br />
<blockquote class="tr_bq">
<span style="font-family: "Courier New",Courier,monospace;"><i>List</i><String> listToJoin = ...</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><i>Monoflop mf = new Monoflop();</i></span><br />
<span style="font-family: "Courier New",Courier,monospace;"><i>StringBuilder</i> result = new <i>StringBuilder</i>();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><b>for</b>(<i>String</i> item : listToJoin) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <b>if</b> (mf.successiveCall()) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> result.append(", ");</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> result.append(item);</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span>
<span style="font-family: "Courier New",Courier,monospace;"><i>System</i>.out.println(result);</span></blockquote>
Granted, you didn't save a whole lot of line in this example. However the logic is much more visible and you have less possibilities for errors. The only thing which is a bit misleading is that a call named successiveCall() has side-effects. On the otherhad, as it toggles the internal state, I didn't want to make it a getter (isSuccessiveCall()) since that would be even more evil.<br />
<br />
Feel free to use this class in your own code base (but use the one from GitHub - as it is better documented). However, if you like it and you have uses for the fastest dependency injection framework out there, with lots of other features, check out: <a href="http://sirius-lib.net/">http://sirius-lib.net</a> (<a href="https://github.com/scireum/sirius" target="_blank">GitHub</a>). SIRIUS (which contains Monoflop) is OpenSource (MIT license) and developed and maintained by <a href="http://www.scireum.de/" target="_blank">scireum GmbH</a>.<br />
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-32180855383689630072013-09-02T13:29:00.000+02:002013-09-02T13:29:34.171+02:00S3 Emulator / Running S3 Locally<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKmATdeGbf9gmbPMUOLa1BvqMXUymsSUgo88IS4xD3HiiB5Qvw8Rr3ZdJjSwVnogL8vNP2L8P198tum7dbY7T9MVlCw1b-zWWz_1khGrWFNrmz1EuW92gXirPNbqMjWJpg169DGluPqirx/s1600/s3ninja_logo_s3_400_300.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="236" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKmATdeGbf9gmbPMUOLa1BvqMXUymsSUgo88IS4xD3HiiB5Qvw8Rr3ZdJjSwVnogL8vNP2L8P198tum7dbY7T9MVlCw1b-zWWz_1khGrWFNrmz1EuW92gXirPNbqMjWJpg169DGluPqirx/s320/s3ninja_logo_s3_400_300.jpg" width="320" /></a></div>
<br />
We've all been there: You're in a remote spot, distant from all your daily stress and hassle. A romantic candle light setup puts you in the right mood - Nothing can stop you from coding on your newest cloud project like there's no morning. <b>Nothing?</b> Well, without a proper internet connection, accessing S3 and the like is quite unpleasant. But fear no more! With <b><a href="http://s3ninja.net/" target="_blank">S3 ninja</a></b> you can setup a S3 / Ceph compatible API in seconds.<br />
<br />
Download the latest zip from s3ninja.net, follow the installation instrcutions and you're ready to go. Being a java application, you need nothing but an installed Java JRE (1.6.xx). At last a word of caution: S3 ninja is intended to be a simple easy to
use emulator for the S3 API. It has neither caching nor any kind of
replcation layer.<br />
<br />
Feel like hacking? Both, S3 ninja and its underlying server platform <b><a href="http://sirius-lib.net/" target="_blank">SIRIUS</a></b> are open source and welcome contributions: <a href="https://github.com/scireum/s3ninja" target="_blank">Project on Github</a><br />
<br />
This is what the main GUI looks like. On the left it even shows you the credentials to use, to test your hash generation:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIgJjTSq9r7GLf6Ib4G_wOQ69lIz9WJhjB938JC84oVv0qypA9s24hFC7lJhzoKwKzP2PETwTB4d_C5CIJ02n-D_xZ4mvwkOz80Mll6IFWHNG0I9JnWRRG65fYpfuwerhj1vag1iw1Co2w/s1600/ninja-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjIgJjTSq9r7GLf6Ib4G_wOQ69lIz9WJhjB938JC84oVv0qypA9s24hFC7lJhzoKwKzP2PETwTB4d_C5CIJ02n-D_xZ4mvwkOz80Mll6IFWHNG0I9JnWRRG65fYpfuwerhj1vag1iw1Co2w/s400/ninja-1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
In the (API) Acess Logs all calls against the S3 API are tracked and visualized:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrIBUyqtfxx5mD_0mQodIaLyfSH1y-aBOYlBPxAi9gjfSSmcKdBVUoI7cE7m8U0-l4H-PtjnpdxIPF9tpPI-5yUX17is9pXDgufYjwj459piICMlOV8xUNqsI6YP_W7_NZUALZL0DMwwBy/s1600/ninja-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrIBUyqtfxx5mD_0mQodIaLyfSH1y-aBOYlBPxAi9gjfSSmcKdBVUoI7cE7m8U0-l4H-PtjnpdxIPF9tpPI-5yUX17is9pXDgufYjwj459piICMlOV8xUNqsI6YP_W7_NZUALZL0DMwwBy/s400/ninja-2.png" width="400" /></a></div>
<br />
<span id="goog_1160084587"></span><span id="goog_1160084588"></span><br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com0tag:blogger.com,1999:blog-4531716331336679054.post-52068274526353396182012-12-19T10:15:00.000+01:002012-12-19T10:16:18.078+01:00Generating Barcodes in PDFs with Flying-Saucer<a href="http://code.google.com/p/flying-saucer/">Flying-Saucer</a> is a nice library to generate PDF documents from within Java applications. Just generate a bunch of XHTML, throw it into the renderer and let it produce the desired document utilizing iText.<br />
<br />
When it comes to barcodes however, Flying-Saucer cannot access the built in barcode functionality of iText (at least I didn't find any documentation for it).<br />
<br />
However, being OpenSource and well designed, one only needs to create one subclass to achieve the task: Flying-Saucer relies on a factory named <span style="font-family: "Courier New",Courier,monospace;">ReplacedElementFactory</span>, which can replace elements by custom objects. This is also used to embed images, as the class <span style="font-family: "Courier New",Courier,monospace;">ITextReplacedElementFactory</span> shows. Now we can simply create a subclass, which replaces images with an appropriate barcode: <b><span style="font-family: "Courier New",Courier,monospace;"><br /></span></b><br />
<b><span style="font-family: "Courier New",Courier,monospace;"></span></b><br />
<b><span style="font-family: "Courier New",Courier,monospace;"><img src="0123456789" type="code128" style="height: 1cm" /></span></b><br />
<br />
<span style="font-family: inherit;">One simply needs to override the createReplacedElement method like this (the whole code can be found here: <a href="https://github.com/andyHa/scireumOpen/blob/master/src/com/scireum/open/fs/BarcodeReplacedElementFactory.java" target="_blank">BarcodeReplacedElementFactory.java (GitHub)</a>):</span><b><span style="font-family: "Courier New",Courier,monospace;"> </span></b><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">@Override<br /> public ReplacedElement createReplacedElement(</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> LayoutContext c, </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> BlockBox box,<br /> UserAgentCallback uac, </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> int cssWidth, </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> int cssHeight) </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> {<br /><br /> Element e = box.getElement();<br /> if (e == null) {<br /> return null;<br /> }<br /><br /> String nodeName = e.getNodeName();<br /> if (nodeName.equals("img")) {<br /> if ("code128".equals(e.getAttribute("type"))) {<br /> try {<br /> Barcode128 code = new Barcode128();<br /> code.setCode(e.getAttribute("src"));<br /> FSImage fsImage = new ITextFSImage(</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> Image.getInstance(<br /> code.createAwtImage(</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> Color.BLACK, </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> Color.WHITE</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ),<br /> Color.WHITE</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ));<br /> if (cssWidth != -1 || cssHeight != -1) {<br /> fsImage.scale(cssWidth, cssHeight);<br /> }<br /> return new ITextImageElement(fsImage);<br /> } catch (Throwable e1) {<br /> return null;<br /> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }<br /> }<br /><br /> return super.createReplacedElement(</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> c, box, uac, cssWidth, cssHeight);<br /> }</span><br />
<br />
Granted, "type" is no valid XHTML-Element for <img /> but as you can see in the code above, you could easily replace it with data-type or any other attribute. Flying-Saucer doesn't seem to care about this anyway.<br />
<br />
Note: The code above can only handle Code128-Barcodes, but can easily be extended to handle EAN and the like (iText supports a whole bunch of barcodes by default).<br />
<br />
In order to make our factory work, we need to pass it to the renderer - which is pretty darn easy:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> ITextRenderer renderer = new ITextRenderer();<br /> renderer.getSharedContext().setReplacedElementFactory(<br /> new BarcodeReplacedElementFactory(</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> renderer.getOutputDevice()</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> ));<br /> renderer.setDocumentFromString(inputAsString);<br /> renderer.layout();<br /> renderer.createPDF(outputAsStream);</span>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1tag:blogger.com,1999:blog-4531716331336679054.post-54120223890705757872012-02-09T11:37:00.000+01:002012-02-09T11:37:00.020+01:00Keyboard Navigation between Input Fields using JQueryIf you have a form with several input fields, one can easily add keyboard based navigation using JQuery. This means, that if the user is in the 1st input field and pushes the down key, the focus jumps to the next field, etc.<br />
<br />
To achive this, we define two helper functions, which find the next or the previous instance for a given selected range:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">jQuery.fn.elementAfter = function(other) {<br /> for(i = 0; i < this.length - 1; i++) {<br /> if (this[i] == other) {<br /> return </span><span style="font-family: "Courier New",Courier,monospace;">jQuery</span><span style="font-family: "Courier New",Courier,monospace;">(this[i + 1]);<br /> }<br /> }<br /> return jQuery;<br />};<br /><br />jQuery.fn.elementBefore = function(other) {<br /> if (this.length > 0) { <br /> for(i = 1; i &lt; this.length; i++) {<br /> if (this[i] == other) {<br /> return </span><span style="font-family: "Courier New",Courier,monospace;">jQuery</span><span style="font-family: "Courier New",Courier,monospace;">(this[i - 1]);<br /> }<br /> }<br /> }<br /> return jQuery;<br />};</span><br />
<br />
To use this with <span style="font-family: "Courier New",Courier,monospace;">input </span>fields, we can add the following in the <span style="font-family: "Courier New",Courier,monospace;">jQuery(document).ready</span> function:<br />
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<span style="font-family: "Courier New",Courier,monospace;">$('input').bind('keyup', function(evt) {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> if (evt.keyCode == 40) {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> // down key</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> $('input').<b>elementAfter</b>(this).focus();</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> } else if (evt.keyCode == 38) {</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> // up key</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> $('input').<b>elementBefore</b>(this).focus();</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;"> }</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">});</span><br />
<br />
There you go, once the focus is in any input field, you can easily navigate to the previous/next one, using the up/down keys.andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com4tag:blogger.com,1999:blog-4531716331336679054.post-35376912227710664412012-02-06T11:28:00.006+01:002012-02-06T14:42:05.312+01:00QSyntaxHighlighter - Colorized Braces<div class="separator" style="clear: both; text-align: center;">
</div>
As I've written in <a href="http://andreas.haufler.info/2012/02/better-syntax-coloring-for-ides.html">one of my previous posts</a> I like the idea to color subsequent braces differently to improve the overall readability.<br />
<br />
Hacking this into <a href="http://developer.qt.nokia.com/doc/qt-4.8/qsyntaxhighlighter.html">QSyntaxHighlighter</a> is, as almost everything in Qt, quite easy. I created a struct, which saves the number of braces, left open for each block (paragraph):<br />
<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">struct BlockData : public QTextBlockUserData</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">{</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> int numBraces;</span><br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">};</span><br />
<span class="Apple-style-span" style="font-family: inherit;">Within the highlightBlock method, one can easily access this information, process all braces of the current paragraph, and save the number of open braces:</span><br />
<pre style="font: normal normal normal 12px/normal 'Bitstream Vera Sans Mono', Courier, monospace; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">void Highlighter::highlightBlock(const QString &text)
{
BlockData* data = dynamic_cast<BlockData*>(
currentBlock().previous().userData());
int numberOfOpenBraces = 0;
if (data != NULL) {
numberOfOpenBraces = data->numBraces;
}
..tokenizing...update numOfOpenBraces..
BlockData* newData = dynamic_cast<BlockData*>(currentBlockUserData());
if (newData == NULL) {
newData = new BlockData();
}
newData->numBraces = numberOfOpenBraces;
setCurrentBlockUserData(newData);
}</span></pre>
<pre style="font: normal normal normal 12px/normal 'Bitstream Vera Sans Mono', Courier, monospace; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"></pre>
<pre style="font: normal normal normal 12px/normal 'Bitstream Vera Sans Mono', Courier, monospace; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"></pre>
<pre style="font: normal normal normal 12px/normal 'Bitstream Vera Sans Mono', Courier, monospace; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"></pre>
<pre style="font: normal normal normal 12px/normal 'Bitstream Vera Sans Mono', Courier, monospace; margin-bottom: 0px; margin-left: 0px; margin-right: 0px; margin-top: 0px; padding-bottom: 0px; padding-left: 0px; padding-right: 0px; padding-top: 0px;"></pre>
<span class="Apple-style-span" style="font-family: inherit;">This is an example expression, without coloring or braces:</span><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnMW9HCyvRCuMuLu_T-_DTU3irh0xnP7WdRQFZPn_sTkkC83CQF39PJEN39SCSVo4LVSpVV9Njf0AAvESeAtQLui7WEy7_qQ6IxjmZLvI9IF16uc3DcQCGRCOK5NnUViYZhZXEuy8iO_6n/s1600/oldlinecolor.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnMW9HCyvRCuMuLu_T-_DTU3irh0xnP7WdRQFZPn_sTkkC83CQF39PJEN39SCSVo4LVSpVV9Njf0AAvESeAtQLui7WEy7_qQ6IxjmZLvI9IF16uc3DcQCGRCOK5NnUViYZhZXEuy8iO_6n/s1600/oldlinecolor.png" /></a></div>
<br />
This is the same example, with coloring: (Yes, I might want to change the colors, I know...)<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF6OdOwxWFSorHzjVUUUmxTR0-fEvS3ypakjR1VF-oIkFyGhfRESEPu4KG19ljMQoE0n9UYbyp90rqrgdJQ2utrMMmBPTdz64c1cAmG_LdfL-U9XnP32gkNw0Pl1qSI9qQnV4EMiteqCW6/s1600/newlinecolor.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiF6OdOwxWFSorHzjVUUUmxTR0-fEvS3ypakjR1VF-oIkFyGhfRESEPu4KG19ljMQoE0n9UYbyp90rqrgdJQ2utrMMmBPTdz64c1cAmG_LdfL-U9XnP32gkNw0Pl1qSI9qQnV4EMiteqCW6/s1600/newlinecolor.png" /></a></div>
<br />
The complete source code can be found here: <a href="https://github.com/andyHa/pimii/blob/master/gui/highlighter.cpp">Highlighter.cpp</a>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com2tag:blogger.com,1999:blog-4531716331336679054.post-52868794298158449352012-02-02T19:11:00.000+01:002012-02-06T18:12:32.200+01:00Better Syntax Coloring for IDEsModern IDEs like Eclipse already provide excellent syntax highlighting. Still I think there is room for improvement.<br />
<br />
One idea of my ideas would be to color brackets differently. Instead of
drawing them all in the same color, rotating through a color wheel would
make it much easier to check, where which bracket is closed.<br />
<br />
<b>So this example code:</b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG7rivXEjOQ7GTYOueVP5lqWbpuTrpwBvIuCg9yb25RQdN7XuvzeZ__jtj6sGczxU7SAqyQgW4UNF-urYAkVY5bD0XedLN9d5S5H9Iu6EFncFWf3fFGUgqa2Omw4zKUEyL-iGiBzROKNk5/s1600/syntax_old.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiG7rivXEjOQ7GTYOueVP5lqWbpuTrpwBvIuCg9yb25RQdN7XuvzeZ__jtj6sGczxU7SAqyQgW4UNF-urYAkVY5bD0XedLN9d5S5H9Iu6EFncFWf3fFGUgqa2Omw4zKUEyL-iGiBzROKNk5/s1600/syntax_old.png" /></a></div>
<div class="" style="clear: both; text-align: left;">
<b>Would look like this:</b> </div>
<div class="" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWqyfullcwBxIEhM0F9lIaJ3kKNYJJYbOE_nya5E9BqB-XmxX7YP8f5R5wO2waKtN-I3nKveh8xI3ulEpLFGv2BmCS8FELEMexgYvr6MkgE_2RuSu1htfzbkbmIFNvCcolqu75-Aopyyv3/s1600/syntax_new.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiWqyfullcwBxIEhM0F9lIaJ3kKNYJJYbOE_nya5E9BqB-XmxX7YP8f5R5wO2waKtN-I3nKveh8xI3ulEpLFGv2BmCS8FELEMexgYvr6MkgE_2RuSu1htfzbkbmIFNvCcolqu75-Aopyyv3/s1600/syntax_new.png" /></a></div>
<br />
<div class="" style="clear: both; text-align: left;">
<br />
Of course it's a detail, but a visual cue like this can help when you would try to extract the inner expression in FTFilter.notAnalyzed - Just start your selection from the green bracket, to the closing green bracket and you're done.</div>
<div class="" style="clear: both; text-align: left;">
<br /></div>
<div class="" style="clear: both; text-align: left;">
I already downloaded the JDT sources of Eclipse, but in order to have a fast and precise syntax coloring, the code is quite complex and I don't have the time to work into this. Still I believe a JDT developer could implement this feature very quickly - if you happen to know one, you might want to forward the URL ;-)</div>
<br />
If you think this is a good idea, vote me up on DZone, or retweet this or +1 it on google... If there is enough positive feedback I'll go and file a bug for Eclipse, maybe it'll be in one of the next versions...<br />
<br />
<b>Update 1: </b>I received a lot of positive feedback, thanks everybody. Three other programs seem to already support this feature: Microsoft Excel, Pharo Smalltalk (I think, only when the cursor is near a brace) and the Closure Plugin for Eclipse.<br />
<br />
Additionally I hacked this functionality into a subclass of QSyntaxHighlighter for one of my Qt toy projects. As <a href="http://andreas.haufler.info/2012/02/qsyntaxhighlighter-colorized-braces.html">I've written here</a>, this is quite simple.<br />
<br />
<b>Update 2: </b>Just filed a Bug for Eclipse JDT-Text: <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=370745">https://bugs.eclipse.org/bugs/show_bug.cgi?id=370745</a>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com8tag:blogger.com,1999:blog-4531716331336679054.post-61117614615498885752012-01-23T14:30:00.000+01:002012-02-06T21:23:25.806+01:00Modular Java Applications - A Microkernel ApproachSoftware Engineering is all about <b>reuse</b>. We programmers therefore love to split applications up into smaller components so that each of them can be reused or extended in an independent manner.<br />
<br />
A keyword here is "<b>loose coupling</b>". Slightly simplified, this means, each component should have as few dependencies to other components as possible. Most important, if I have a component B which relies on component A, I don't want that A needs to know about B. The component A should just provide a clean interface which could be used and extended by B.<br />
<br />
In Java there are many frameworks which provide this exact functionality: <a href="http://www.oracle.com/technetwork/java/javaee/overview/index.html">JavaEE</a>, <a href="http://www.springsource.org/">Spring</a>, <a href="http://www.osgi.org/Main/HomePage">OSGI</a>. However, each of those frameworks come with their own way to do things and provide lots and lots of additional functionality - whether you want it or not!<br />
<br />
Since we here at <a href="http://www.scireum.de/">scireum </a>love modularity (we build 4 products out of a set of about 10 independet modules) we built our own little framework. I factored out the most important parts and now have a single class with less than 250 lines of code+comments!<br />
<br />
I call this a microkernel approach, since it nicely compares to the situation we have with operating systems: There are<a href="http://en.wikipedia.org/wiki/Monolithic_kernel"> monolithic kernels</a> like the one of Linux with about <a href="http://en.wikipedia.org/wiki/Linux_%28kernel%29">11,430,712</a> lines of code. And there is a concept called a <a href="http://en.wikipedia.org/wiki/Microkernel">microkernel</a>, like to one of Minix with about <a href="http://en.wikipedia.org/wiki/MINIX_3#Reduce_kernel_size">6,000</a> lines of executable kernel code. There is still an ongoing discussion which of the two solituons is better. A monolithic kernel is faster, a microkernel has way less critical code (critical code means: a bug there will crash the complete system. If you haven't already, you should read more about mikrokernels <a href="http://en.wikipedia.org/wiki/Microkernel">on Wikipedia</a>.<br />
<br />
However one might think about operating systems - when it comes to Java I prefer less dependencies and if possible no black magic I don't understand. Especially if this magic involves complex <span style="font-family: "Courier New",Courier,monospace;">ClassLoader </span>structures. Therefore, here comes <span style="font-family: "Courier New",Courier,monospace;">Nucleus</span>...<br />
<br />
<b><span style="font-size: large;">How does this work?</span></b><br />
The framework (<span style="font-family: "Courier New",Courier,monospace;">Nucleus) </span>solves two problems of modular applications:<br />
<ul>
<li>I want to provide a service to other components - but I only want to show an interface and they should be provided with my implementation at runtime without knowning(referencing) it.</li>
<li>I want to provide a service or callback for other components. I provide an interface, and I want to know all classes implementig it, so I can invoke them.</li>
</ul>
Ok, we probably need examples for this. Say we want to implement a simple timer service. It provides an interface:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">public interface EveryMinute {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> void runTimer() throws Exception;</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span><br />
<br />
All classes implementing this interface should be invoked every minute. Additionally we provide some infos - namely, when was the timer executed last.<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
public interface TimerInfo {<br />
String getLastOneMinuteExecution();<br />
}</div>
<br />
Ok, next we need a client for our services:<br />
<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
@Register(classes = EveryMinute.class)<br />
public class ExampleNucleus implements EveryMinute {</div>
<div style="font-family: "Courier New",Courier,monospace;">
<br />
private static Part<TimerInfo> timerInfo = </div>
<div style="font-family: "Courier New",Courier,monospace;">
Part.of(TimerInfo.class);<br />
<br />
public static void main(String[] args) throws Exception {<br />
Nucleus.init(); </div>
<div style="font-family: "Courier New",Courier,monospace;">
<br />
while (true) {<br />
Thread.sleep(10000);</div>
<div style="font-family: "Courier New",Courier,monospace;">
System.out.println("Last invocation: "<br />
+ timerInfo.get().getLastOneMinuteExecution());<br />
}<br />
<br />
}<br />
<br />
@Override<br />
public void runTimer() throws Exception {<br />
System.out.println("The time is: "<br />
+ DateFormat.getTimeInstance().format(new Date()));<br />
}<br />
}</div>
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<div style="font-family: inherit;">
The static field "<span style="font-family: "Courier New",Courier,monospace;">Part<TimerInfo> timerInfo</span>" is a simple helper class which fetches the registered instance from <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>on the first call and loads it into a private field. So accessing this part has almost no overhead to a normal field access - yet we only reference an interface, not an implementation.</div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
The <span style="font-family: "Courier New",Courier,monospace;">main </span>method first initializes <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>(this performs the classpath scan etc.) and then simply goes into an infinite loop, printing the last execution of our timer every ten seconds.</div>
<div style="font-family: inherit;">
<br /></div>
<div style="font-family: inherit;">
Since our class wears a <span style="font-family: "Courier New",Courier,monospace;">@Register</span> annotation, it will be discovered by a special <span style="font-family: "Courier New",Courier,monospace;">ClassLoadAction</span> (not by <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>itself) instantiated and registered for the <span style="font-family: "Courier New",Courier,monospace;">EveryMinute</span> interface. Its method <span style="font-family: "Courier New",Courier,monospace;">runTimer </span>will then be invoced by our timer service every minute.</div>
<br />
Ok, but how would our TimerService look like?<br />
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<div style="font-family: "Courier New",Courier,monospace;">
@Register(classes = { TimerInfo.class })<br />
public class TimerService implements TimerInfo {<br />
<br />
@InjectList(EveryMinute.class)<br />
private List<EveryMinute> everyMinute;<br />
private long lastOneMinuteExecution = 0;<br />
<br />
private Timer timer;<br />
<br />
public TimerService() {<br />
start();<br />
}<br />
<br />
public void start() {<br />
timer = new Timer(true);</div>
<div style="font-family: "Courier New",Courier,monospace;">
// Schedule the task to wait 60 seconds and then invoke</div>
<div style="font-family: "Courier New",Courier,monospace;">
// every 60 seconds.<br />
timer.schedule(new InnerTimerTask(), </div>
<div style="font-family: "Courier New",Courier,monospace;">
1000 * 60, </div>
<div style="font-family: "Courier New",Courier,monospace;">
1000 * 60);<br />
}</div>
<span style="font-family: "Courier New",Courier,monospace;"> private class InnerTimerTask extends TimerTask {</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> @Override</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> public void run() {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> // Iterate over all instances registered for</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> // EveryMinute and invoke its </span><span style="font-family: "Courier New",Courier,monospace;">runTimer </span><span style="font-family: "Courier New",Courier,monospace;">method.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> for (EveryMinute task : everyMinute) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> task.runTimer();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
// <span style="font-family: "Courier New",Courier,monospace;">Update lastOneMinuteExecution </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> lastOneMinuteExecution = System.currentTimeMillis();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> @Override</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> public String getLastOneMinuteExecution() {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if (lastOneMinuteExecution == 0) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> return "-";</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> return DateFormat.getDateTimeInstance().format(</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> new Date(lastOneMinuteExecution));</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;">}</span><br />
<br />
<br />
<br />
<br />
This class also wears a <span style="font-family: "Courier New",Courier,monospace;">@Register</span> annotation so that it will also be loaded by the <span style="font-family: "Courier New",Courier,monospace;">ClassLoadAction<span style="font-family: inherit;"></span></span> named above (the <span style="font-family: "Courier New",Courier,monospace;">ServiceLoadAction</span> actually). As above it will be instantiated and put into <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>(as implementation of <span style="font-family: "Courier New",Courier,monospace;">TimerInfo</span>). Additionally it wears an <span style="font-family: "Courier New",Courier,monospace;">@InjectList </span>annotation on the <span style="font-family: "Courier New",Courier,monospace;">everyMinute </span>field. This will be processed by another class named <span style="font-family: "Courier New",Courier,monospace;">Factory </span>which performs simple dependency injection. Since its constructur starts a Java <span style="font-family: "Courier New",Courier,monospace;">Timer </span>for the <span style="font-family: "Courier New",Courier,monospace;">InnerTimerTask</span>, from that point on all instances registered for <span style="font-family: "Courier New",Courier,monospace;">EveryMinute </span>will be invoced by this timer - as the name says - every minute.<br />
<br />
<b><span style="font-size: large;">How is it implemented?</span></b><br />
The good thing about <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>is, that it is powerful on the one hand, but very simple and small on the other hand. As you could see, there is no inner part for special or privileged services. Everything is built around the kernel - the class <span style="font-family: "Courier New",Courier,monospace;">Nuclues</span>. Here is what it does:<br />
<br />
<ul>
<li>It scans the classpath and looks for files called "<span style="font-family: "Courier New",Courier,monospace;">component.properties</span>". Those need to be in the root folder of a JAR or in the /src folder of each Eclipse project respectively. </li>
<li>For each identified JAR / project / classpath element, it then collects all contained class files and loads them using <span style="font-family: "Courier New",Courier,monospace;">Class.forName</span>.</li>
<li>For each class, it checks if it implements <span style="font-family: "Courier New",Courier,monospace;">ClassLoadAction</span>, if yes, it is put into a special list.</li>
<li>Each <span style="font-family: "Courier New",Courier,monospace;">ClassLoadAction </span>is instanciated and each previously seen class is sent to it using: <span style="font-family: "Courier New",Courier,monospace;">void handle(Class<?> clazz)</span></li>
<li>Finally each <span style="font-family: "Courier New",Courier,monospace;">ClassLoadAction </span>is notified, that nucleus is complete so that final steps (like annotation based dependency injection) could be performed.</li>
</ul>
That's it. The only other thing <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>provides is a registry which can be used to register and retrieve objects for a class. (An in-depth description of the process above, can be found here: http://andreas.haufler.info/2012/01/iterating-over-all-classes-with.html).<br />
<br />
Now to make this framework useable as shown above, there is a set of classes around <span style="font-family: "Courier New",Courier,monospace;">Nucleus</span>. Most important is the class <span style="font-family: "Courier New",Courier,monospace;">ServiceLoadAction</span>, which will instantiate each class which wears a <span style="font-family: "Courier New",Courier,monospace;">@Register</span> annoation, runs <span style="font-family: "Courier New",Courier,monospace;">Factory.inject </span>(our mini DI tool) on it, and throws it into <span style="font-family: "Courier New",Courier,monospace;">Nucleus </span>for the listed classes. <b>Whats important</b>: The <span style="font-family: "Courier New",Courier,monospace;">ServiceLoadActions </span>has no specific rights or privileges, you can easily write your implementation which does smarter stuff.<br />
<br />
Next to some annotations, there are three other handy classes when it comes to retrieving instances from <span style="font-family: "Courier New",Courier,monospace;">Nucleus</span>: <span style="font-family: "Courier New",Courier,monospace;">Factory, Part </span>and <span style="font-family: "Courier New",Courier,monospace;">Parts</span>. As noted above, the <span style="font-family: "Courier New",Courier,monospace;">Factory </span>is a simple dependency injector. Currently only the ServiceLoadAction autmatically uses the Factory, as all classes wearing the <span style="font-family: "Courier New",Courier,monospace;">@Register</span> annotation are scanned for required injections. You can however use this factory to run injections on your own classes or other <span style="font-family: "Courier New",Courier,monospace;">ClassLoadActions </span>to do the same as <span style="font-family: "Courier New",Courier,monospace;">ServiceLoadAction</span>. If you can't or don't want to rely in annotation based dependency magic, you can use the two helper classes <span style="font-family: "Courier New",Courier,monospace;">Part </span>and <span style="font-family: "Courier New",Courier,monospace;">Parts</span>. Those are used like normal fields (see <span style="font-family: "Courier New",Courier,monospace;">ExampleNucleus.timerInfo</span> above) and fetch the appropriate object or list of objects automatically. Since the result is cached, repeated invocations have almost no overhead compared to a normal field.<br />
<br />
Nucleus and the example shown above is open source (MIT-License) and available here:<br />
<a href="https://github.com/andyHa/scireumOpen/blob/master/src/examples/ExampleNucleus.java">https://github.com/andyHa/scireumOpen/blob/master/src/examples/ExampleNucleus.java</a><br />
<a href="https://github.com/andyHa/scireumOpen/tree/master/src/com/scireum/open/nucleus">https://github.com/andyHa/scireumOpen/tree/master/src/com/scireum/open/nucleus</a><br />
<br />
<br />
If you're interested in using <span style="font-family: "Courier New",Courier,monospace;">Nucleus</span>, I could put the relevant souces into a separater repository and also provide a release jar - just write a comment below an let me know.<br />
<br />
<b>Update: </b>I moved nucleus into a repository on its own: https://github.com/andyHa/nucleus - It even includes a distribution jar.<br />
<br />
<br />
<span style="font-size: xx-small;">This post is the fourth part of the my
series "Enterprisy Java" - We share our hints and tricks how to overcome
the obstacles when trying to build several multi tenant web
applications out of a set of common modules.</span>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com10tag:blogger.com,1999:blog-4531716331336679054.post-38566506751171991612012-01-10T23:14:00.001+01:002012-02-06T21:23:39.364+01:00Launching and Debugging Tomcat from Eclipse without complex plugins Modern IDEs like Eclipse provide various Plugins to ease web
developement. However, I believe that starting Tomcat as "normal" Java
application still provides the best debugging experience. Most of the
time, this is because these tools launch Tomcat or any other servlet
container as external process and then attach a remote debugger on it.
While you're still able to set breakpoints and inspect variables, other
features like hot code replacement don't work that well.<br />
<br />
Therefore I prefer to start my Tomcat just like any other Java application from within Eclipse. Here's how it works:<br />
<br />
<i>This article addresses experienced Eclipse users. You
should already know how to create projects, change their built path and
how to run classes. If you need any help, feel free to leave a comment
or contact me.</i><br />
<br />
We'll add the Tomcat as additional Eclipse project, so that paths and
all remain platform independent. (I even keep this project in our SVN
so that everybody works with the same setup).<br />
<br />
<b>Step 1 </b>- Create new Java project named "<i>Tomcat7</i>"<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlVdB8tiZDWYxH1WsvbfQMy5UABbSvq-ZVA5u393xJwbicNNZNlRTtfuZ4AJITxK9A6oeESVVMNwjshZR2MdjMNzwNQfYxGmRWUjP_kitPP_f4KGxQQXzuMaspwk3czm02WK7BS7gGAv4m/s1600/step1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlVdB8tiZDWYxH1WsvbfQMy5UABbSvq-ZVA5u393xJwbicNNZNlRTtfuZ4AJITxK9A6oeESVVMNwjshZR2MdjMNzwNQfYxGmRWUjP_kitPP_f4KGxQQXzuMaspwk3czm02WK7BS7gGAv4m/s640/step1.png" width="542" /></a></div>
<br />
<br />
<br />
<br />
<b>Step 2 </b>- Remove the "src" source folder<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_KP_i5cwtW4kCm5iWmidR8dTTX-HDuMzYMtaqFuiNAP_YFEAnPEZWarfl3kfEGgeqgUANKOlMqWpsaMHWbruuDGHEsO9DveJgdEbINGSWGeyEOzAcN0r6pXdtidGuSfhOFuWOQ_EebJ_N/s1600/step2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="486" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg_KP_i5cwtW4kCm5iWmidR8dTTX-HDuMzYMtaqFuiNAP_YFEAnPEZWarfl3kfEGgeqgUANKOlMqWpsaMHWbruuDGHEsO9DveJgdEbINGSWGeyEOzAcN0r6pXdtidGuSfhOFuWOQ_EebJ_N/s640/step2.png" width="640" /></a></div>
<br />
<br />
<br />
<b>Step 3</b> - <a href="http://tomcat.apache.org/download-70.cgi">Download Tomcat</a> (Core Version) and unzip into our newly created project. This should now look something like this:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdtaaw42Z1rO7-OzO6e2XoIksAm3jUPQHiYJCjkSlNPhWm_PceBl0DFbOjlDy551NvhsxZKrInq2ZdzPweiwDEzau6RZs7dBU_wnFXtjAlEnPn03B7DT6n9B9O5JVmRvhO6_-ZCjoZaS6u/s1600/step3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="431" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdtaaw42Z1rO7-OzO6e2XoIksAm3jUPQHiYJCjkSlNPhWm_PceBl0DFbOjlDy551NvhsxZKrInq2ZdzPweiwDEzau6RZs7dBU_wnFXtjAlEnPn03B7DT6n9B9O5JVmRvhO6_-ZCjoZaS6u/s640/step3.png" width="640" /></a></div>
<br />
<br />
<br />
<b>Step 4</b> - If you havn't, create a new <i>Test </i>project which
contains your sources (servlets, jsp pages, jsf pages...). Make sure
you add the required libraries to the built path of the project<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgApp3YvXd2HsG5eSkr2EWtuoyj6su7VudwelNDOA2XD2oir2ro5qnXzpztA1nw9ix_ZHnZltfG7nNAg9DVeocK1SOg22aP3xebpWWCeWIhzyec64bqZSl3wwj4trwwCAhiRiKk7i0c6lYz/s1600/step4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="486" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgApp3YvXd2HsG5eSkr2EWtuoyj6su7VudwelNDOA2XD2oir2ro5qnXzpztA1nw9ix_ZHnZltfG7nNAg9DVeocK1SOg22aP3xebpWWCeWIhzyec64bqZSl3wwj4trwwCAhiRiKk7i0c6lYz/s640/step4.png" width="640" /></a></div>
<br />
<br />
<br />
<b> Step 5.1</b> - Create a run configuration. Select our <i>Test </i>project as base and set <b>org.apache.catalina.startup.Bootstrap</b> as main class.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguVmabn2GlXBGSRAG2FlCJ3-IYVer0rqLqMQXAbHYs_90NOfGwfAHc0kQsadndKrEb0IKs2AhFF1421ceyNw5S1dBZgxZ7wrKSQnAQjWORUUpuLp67z_zHQx6RF-2Q1QbVD0XGaGFsDIUY/s1600/step5-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="584" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguVmabn2GlXBGSRAG2FlCJ3-IYVer0rqLqMQXAbHYs_90NOfGwfAHc0kQsadndKrEb0IKs2AhFF1421ceyNw5S1dBZgxZ7wrKSQnAQjWORUUpuLp67z_zHQx6RF-2Q1QbVD0XGaGFsDIUY/s640/step5-1.png" width="640" /></a></div>
<br />
<br />
<br />
<b>Step 5.2 </b>- Optionally specify larger heap settings as VM arguments. <b>Important</b>: Select the "Tomcat" project as working directory (Click on the "Workspace" button below the entry field.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ26kmejrR5wjf3R2EkQvcauMdO3DQWX8EOY2yD65hP2qju42cnbrcdn6_vOQRNtFhkjwMDVgXnsY9iCmBHzKkUCqrG7d4Ut5He0QVKlSTswih9iYSZW_TcB2nYCidkJ5gmzQp00AswCwh/s1600/step5-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJ26kmejrR5wjf3R2EkQvcauMdO3DQWX8EOY2yD65hP2qju42cnbrcdn6_vOQRNtFhkjwMDVgXnsY9iCmBHzKkUCqrG7d4Ut5He0QVKlSTswih9iYSZW_TcB2nYCidkJ5gmzQp00AswCwh/s640/step5-2.png" width="610" /></a></div>
<br />
<br />
<br />
<b>Step 5.3</b> - Add <b>bootstrap.jar </b>and<b> tomcat-juli.jar</b> from the <i>Tomcat7/bin</i> directory as bootstrap classpath.Add everything in <i>Tomcat7/lib </i>as user entries. Make sure the <i>Test</i> project and all other classpath entries (i.e. maven dependencies) are below those.<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjek1BYgNC3HV0odg-szrs2zmfREsKbRp3qHG4yaWKJ_G5j5g4wg-dcBiKtvJeHIFwIa3Pwp0uALsn6K4PNGbJODJ7d9lreVIpMLUyHJrWvFJ4VDqZ5hqbeYuuFoz5alVHzKRBlJNN50lOf/s1600/step5-3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjek1BYgNC3HV0odg-szrs2zmfREsKbRp3qHG4yaWKJ_G5j5g4wg-dcBiKtvJeHIFwIa3Pwp0uALsn6K4PNGbJODJ7d9lreVIpMLUyHJrWvFJ4VDqZ5hqbeYuuFoz5alVHzKRBlJNN50lOf/s640/step5-3.png" width="612" /></a></div>
<br />
<br />
<br />
Now you can "Apply" and start Tomcat by hitting "Debug". After a few seconds (check the console output) you can go to <a href="http://localhost:8080/examples/">http://localhost:8080/examples/</a> and check out the examples provided by Tomcat.<br />
<br />
<br />
<b>Step 6</b> - Add Demo-Servlet - Go to our <i>Test </i>project, add a new package called "demo" and a new servlet called "TestServlet". Be creative with some test output <i>- like I was...</i><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgW77lxUdYQA-tZfAz1uTXS1lqMb6_hxNUXbpUREJzRdoTNa7qqO6oJya2SR78y-7mPLXkJy2pOOgIrwT8pw5HImv3f3nkEnLVBbXUJaQBtg6gTUZ8iTYZp-YXGgtlxOBRljqoTqTFbMqLA/s1600/step6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="550" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgW77lxUdYQA-tZfAz1uTXS1lqMb6_hxNUXbpUREJzRdoTNa7qqO6oJya2SR78y-7mPLXkJy2pOOgIrwT8pw5HImv3f3nkEnLVBbXUJaQBtg6gTUZ8iTYZp-YXGgtlxOBRljqoTqTFbMqLA/s640/step6.png" width="640" /></a></div>
<br />
<br />
Step 7 - Change web.xml - Go to the web.xml of the examples context
and add our servlet (as shown in the image). Below all servlets you also
have to add a servlet-mapping (not shown in the image below). This
looks like that:<br />
<div style="font-family: "Courier New",Courier,monospace;">
<br /></div>
<span style="font-family: "Courier New",Courier,monospace;"> <servlet-mapping></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <servlet-name>test</servlet-name></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> <url-pattern>/demo/test</url-pattern></span><br />
<span style="font-family: "Courier New",Courier,monospace;"> </servlet-mapping></span><br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhS8QmZ7rw0RNEcn1hBP050jGAsKULZo0jpno_9YKq9StARKUIFXZNw2LKsuAWukQEkXRg9OO5MZMN6AzdgxrmSyZeTlNIlLy1U6qbVl-eoTmOkcZmtkiyZqoX0mTHpkW9D8gZGrvBdOKxM/s1600/step7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="548" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhS8QmZ7rw0RNEcn1hBP050jGAsKULZo0jpno_9YKq9StARKUIFXZNw2LKsuAWukQEkXRg9OO5MZMN6AzdgxrmSyZeTlNIlLy1U6qbVl-eoTmOkcZmtkiyZqoX0mTHpkW9D8gZGrvBdOKxM/s640/step7.png" width="640" /></a></div>
<br />
<br />
Hit save and restart tomcat. You should now see your debug output by surfing to <a href="http://localhost:8080/examples/demo/test">http://localhost:8080/examples/demo/test</a>
- You now can set breakpoints, change the output (thanks to hot code
replacement) and do all the other fun stuff you do with other debugging
sessions.<br />
<br />
<br />
<b>Hint:</b> <i>Keeping your JSP/JSF files as well as your web.xml and other resources already in another project?</i>
Just create a little ANT script which copies them into the webapps
folder of the tomcat - and you get re-deployment with a single mouse
click. Even better (this is what we do): You can modify/override the
ResourceResolver of JSF. Therefore you can simply use the classloader to
resolve your .xhtml files. This way, you can keep your Java sources and
your JSF sources close to each other. I will cover that in another post
- The fun stuff starts when running multi tenant systems with custom
JSF files per tenant. The JSF implementation of Sun/Oracle has some nice
gotchas built-in for that case ;-)<br />
<br />andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com6tag:blogger.com,1999:blog-4531716331336679054.post-45125381217566898042012-01-06T16:00:00.000+01:002012-02-06T21:23:54.358+01:00Iterating over all Classes with an Annotation or Interface...is impossible unless you use JavaEE, <b>right?</b> <b>Wrong!</b><br />
<br />
With some tricks you can iterate over all classes which fullfill a given predicate, like implementing an interface or wearing an annotation. But why should you care?<br />
<br />
Well, software engineering is all about reuse. For example we split our software up in about five modules and build currently four different products on top of that. Since a module obviously has no knowledge about any of there products and their classes, we need to discover extensions and handlers and other hooks at runtime.<br />
<br />
Yes, we could use <a href="http://en.wikipedia.org/wiki/OSGi">OSGI </a>or <a href="http://www.springsource.org/">Spring </a>for that, but when we started out, but we both of these "feature battleships" are far too large for our concerns. So we built or own little DI (dependency injection) framework (with about a handful of classes). Well DI is probably not the key aspect, it's actually all about getting all classes implemeting a given interface or annotation. (Some concrete examples will follow in the next posts).<br />
<br />
So how do we get this magic list? Well, it's tricky - but work's like a charm in our setting:<br />
<br />
As I said, we have several modules which each will participate in the seach for classes. Therefore each module will become a JAR file. Now what we do is, we place a file called "component.properties" in the root folder of this JAR, also in the root folder of the Eclipse project repsectively. This file contains some meta-data like name, version and build-date (filled by ant) - but that's irrelevant now.<br />
<br />
Now when we want to discover our classes, we first get a list of all component.properties in the classpath, using the technique above, there will be one per module/JAR:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">Enumeration<URL> e = Nucleus.class.getClassLoader().</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> getResources("component.properties");</span><br />
<br />
We then use each of the returned URLs and apply the following (dirty) algorithm:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> /**</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> * Takes a given url and creates a list which contains </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> * all children of the</span> <span style="font-family: "Courier New",Courier,monospace;">given url. </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> * (Works with Files and JARs).</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> */</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> public static List<String> getChildren(URL url) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> List<String> result = new ArrayList<String>();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if ("file".equals(url.getProtocol())) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> File file = new File(url.getPath());</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> if (!file.isDirectory()) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> file = file.getParentFile();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> addFiles(file, result, file);</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> } else if ("jar".equals(url.getProtocol())) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> try {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> JarFile jar = ((JarURLConnection)<br /> url.openConnection())</span><span style="font-family: "Courier New",Courier,monospace;">.getJarFile();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> Enumeration<JarEntry> e = jar.entries();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> while (e.hasMoreElements()) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> JarEntry entry = e.nextElement();</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> result.add(entry.getName());</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> } catch (IOException e) {</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> Log.UTIL.WARN(e);</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> return result;</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> }</span><br />
<br />
<span style="font-family: "Courier New",Courier,monospace;"> /**</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> * Collects all children of the given file into the given </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> * result list. The resulting string is the relative path</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> * from the given reference.</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> */</span><br />
<span style="font-family: "Courier New",Courier,monospace;"> private static void addFiles(File file, </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> List<String> result, </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> File reference) </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> {<br /> if (!file.exists() || !file.isDirectory()) {<br /> return;<br /> }<br /> for (File child : file.listFiles()) {<br /> if (child.isDirectory()) {<br /> addFiles(child, result, reference);<br /> } else {<br /> String path = null;<br /> while (child != null && !child.equals(reference)) {<br /> if (path != null) {<br /> path = child.getName() + "/" + path;<br /> } else {<br /> path = child.getName();<br /> }<br /> child = child.getParentFile();<br /> }<br /> result.add(path);<br /> }<br /> }<br /> } </span><br />
<br />
<div style="font-family: inherit;">
So what we now have is a list of files like:</div>
<div style="font-family: inherit;">
com/acme/MyClass.class</div>
<div style="font-family: inherit;">
com/acme/resource.txt</div>
...<span style="font-family: "Courier New",Courier,monospace;"> </span><br />
<span style="font-family: "Courier New",Courier,monospace;"> </span><br />
<div style="font-family: inherit;">
We now can filter this list for classes, and load each one, we then can check our predicates (implements interface, has annotation):</div>
<br />
<span style="font-family: "Courier New",Courier,monospace;">...iterating over each result of getChildren(url)...:</span><br />
<span style="font-family: "Courier New",Courier,monospace;"><br /></span><br />
<div style="font-family: "Courier New",Courier,monospace;">
if (relativePath.endsWith(".class")) {</div>
<div style="font-family: "Courier New",Courier,monospace;">
// Remove .class and change / to .<br />
String className = relativePath.substring(0,<br />
relativePath.length() - 6).replace("/", ".");<br />
try {<br />
Class<?> clazz = Class.forName(className);<br />
for (ClassLoadAction action : actions) {</div>
<div style="font-family: "Courier New",Courier,monospace;">
action.handle(clazz);<br />
}<br />
} catch (ClassNotFoundException e) {<br />
System.err.println("Failed to load class: " + className);<br />
} catch (NoClassDefFoundError e) {<br />
System.err.println("Failed to load dependend class: " +</div>
<div style="font-family: "Courier New",Courier,monospace;">
className);<br />
} </div>
<span style="font-family: "Courier New",Courier,monospace;">}</span><br />
<br />
Now you have loaded all classes which match your predicates. We do this once on startup and fill a lookup <span style="font-family: "Courier New",Courier,monospace;">Map</span>. Once a component wants to know all implementations of X, we simply query this Map. Furthermore we use the <span style="font-family: "Courier New",Courier,monospace;">component.properties</span> to let the framework know which component depends on which. We then load the classes in the correct order. This is important, since while loading classes you can provide more implementations of <span style="font-family: "Courier New",Courier,monospace;">ClassLoadAction </span>- <b>Yes, </b>you can extend the extension loader while loading extensions.<br />
<br />
I've made the framework open source. An article about it can be found here: http://andreas.haufler.info/2012/01/modular-java-applications-microkernel.html <br />
<br />
<span style="font-size: xx-small;">This post is the third part of the my
series "Enterprisy Java" - We share our hints and tricks how to overcome
the obstacles when trying to build several multi tenant web
applications out of a set of common modules.</span>andyHahttp://www.blogger.com/profile/10560944830858769783noreply@blogger.com1