Tuesday, July 07, 2015

Running OkHttpClient from within WildFly 9 (subsystem)

A few days ago WildFly 9 was released and one of the highlight for sure is the support of HTTP/2.0 in the Undertow web subsystem. As Hawkular has recently moved to use WildFly 9 (from 8.2) as its underlying server, it was sort of natural to try to use http2 for connections from the Hawkular-Wildfly-Monitor client to the server.

One peculiarity here is that in my case the monitor client is running inside the Hawkular server, but at the end it does not matter if it is running inside a standalone WildFly server or inside the Hawkular server.

The setup

Greg Autric has written a blog post, that shows how to set up Http2 in WildFly with the offline CLI, which also works well in the Hawkular case. As the question came up: that setup also includes the https setup inside WildFly.

The only thing that is a bit problematic in the post is that setting JAVA_OPTS before starting the server will ignore all the settings from standalone.conf, which in the current Hawkular version will prevent a correct start of the bus broker (because the IPv4Only flag is lost).

So in my opinion it is better to modify standalone.conf to *add* those options to the other options that are already there:

  JAVA_OPTS="-Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true"
  JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/p:/opt/hawkular-1.0.0.Alpha3-SNAPSHOT/alpn-boot-8.1.3.v20150130.jar"
  JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"

Now when I start the Hawkular server and try to connect with FireFox on the https port, I get the warning about the self signed certificate, but can connect and get the UI over a Http2 connection as described in Greg's post.

Running the OkHttpClient

As said before, the WildFly monitor client is a subsystem inside the WildFly server. I wrote a bit of client code, that is running in the subsystem (shortened):

   OkHttpClient httpClient;
   httpClient = new OkHttpClient();

   // DO NOT USE IN PRODUCTION, allow all hostnames
   httpClient.setHostnameVerifier(new NullHostNameVerifier());

   setKeystore(httpClient); // Use custom ssl factory

   String uri = "https://...:8443/";

   Request request = new Request.Builder()
            .addHeader("Accept", "application/json")

   // sync execution just for the post
   Response resp = httpClient.newCall(request).execute();


This code works well except for the fact that is always uses Http(s)/1.1 and never Http2 as you can see from the output of the last println statement:

  Response{protocol=http/1.1, code=204, message=....}
I was playing around with various options up to a point where I thought, I have to extract the code to a standalone Java SE class to better debug it in isolation.

I wrote the class, set the bootclasspath, ran it and it worked perfectly:

  Response{protocol=h2, code=204, message=....}

So what is the difference? I removed the bootclasspath setting for ALPN, reran and the connection fell back to http/1.1.

Which is kinda strange as my client subsystem is running inside the very same WilFly server, that is running Undertow and which is able to serve http2 requests and where I added the ALPN classes through JAVA_OPTS earlier.

Now remember that WildFly is using their own classloader system (jboss-modules), which is pretty powerful in isolating deployments and classes and restricting their visibility and/or leakage into areas where they should (not) be seen.

And this in fact is what happened here as well.


So I had to explicitly add the ALPN classes to my module.xml file for the monitoring client subsystem:

  <module xmlns="urn:jboss:module:1.3" name="${moduleName}">
      <resource-root path="clients-common.jar"/>
      <resource-root path="okhttp.jar"/>
      <resource-root path="okio.jar"/>
      <!-- modules required by any subsystem -->
      <module name="javax.api"/>
      <system export="true">
          <!-- Needed for HTTP2 and SPDY support-->
          <path name="org/eclipse/jetty/alpn"/>

From the above snippet, you can see that the okhttp and okio jars are packaged in the module and are made available to my client code as well.

Now that the module.xml has been adjusted, as is well and my subsystem is using Http2 :-)

Wednesday, May 13, 2015

Scripts with multiple actions in Textual5

Some may know Textual as a powerful IRC client on the Mac.

One of the interesting parts (to me) is that Textual allows to extend it via Scripts, that can be written in AppleScript or other scripting languages.

They have an introuction to write Scripts, which I've used to create some simple scripts.

What I always wanted to do it to have scripts that can do multiple things like

/away Gone fishing
/msg I'am out of here

To set myself into away mode, put a message in the channel and then leave it.

Unfortunately, this seemed impossible.

Yesterday I finally emailed their support and got an answer back that this is pretty simple: just put each command on a new line.

So a potential script could look like this:

on textualcmd(inputData, destinationChannel)

return "
/away Gone for the night
/nick zzZZzz
end textualcmd

to set may away mode and also change my nick. It is important to put each of the
command on the very first column of the line, as Textual does not remove leading spaces.

Two helpful links: Textual Command reference, list of IRC commands on Wikipedia.

UPDATE: the "WritingScripts" article meanwhile got updated to better reflect above use case.

Monday, February 23, 2015

Little extension board for the PI

Long time ago I've started creating a small extension board for the RaspberryPI to get some physical world into hacking.

Img of installed and running board
Board installed and running on the PI.

The first iteration was on a bread board with some wiring, then for the next iteration I took a stripe PCB and wired it there. Worked but was not pretty:

Prototype board

Anyway, as the concept worked out, I decided I need some printed circuits. I played around with some options, but nothing was really cool for hobbyists (and I did not want to manufacture them my self with all the chemicals needed). And then I found Fritzing, which was exactly right.

Not only does Fritzing produce boards, but it also a nice graphical editor application that can be used to create a schematic, assign parts from a large part library, (auto) route the schematic and send the result to production

Fritzing editor with PCB layout
Fritzing editor with routed PCB in both side view

After I created the layout etc. I sent the design to the Fritzing fab and got by PCBs back after around a week of round trip time.

Yesterday I now sat down with the PCB, some needed parts and my soldering iron and assembled the parts

Parts and PCB
The parts
After soldering the low profile parts
Low profile parts soldered in
Most parts soldered

In the last image there was still the thermo sensor missing, as well as the 4th LED.

Now when that board is put on to the extension header of the PI (inner row, that is closer to the CPU, the following little shell script will light all LEDs and then display the current temperature every second

set -x
cd /sys/class/gpio
for i in 10 22 27
echo $i > export
sleep 5
echo out > gpio$i/direction
sleep 5
echo 1 > gpio$i/value

cd /sys/bus/w1/devices/10-*
while true
cat w1_slave | grep t=
sleep 60

There is one line that may need tailoring depending on the variant of DS1820 thermometer chip you have

cd /sys/bus/w1/devices/10-*

One thing where I am not yet sure is if the DS1820 actually needs the phantom power. In an older experiment with the chip, I did not connect pin 3 at all. I think this additional power may even heat the thermometer chip, as the values that I currently get a are some 5-6 degrees too high.

To read the state of the push button you can use this script

cd /sys/class/gpio
echo 9 > export
cd gpio9
while true ; do cat value; done

If you are interested about more details of the board, you can look at this Fritzing page, where I've uploaded the .fzz file.

And if you are using RHQ, you can look at this agent plugin to use the LEDs and the thermometer chip from within RHQ.

Friday, February 06, 2015

Meet the Hawk!

Well, actually that is not Hawk, but HAWKULAR

(Non-official visualization)

Hawkular is the next generation monitoring (and management) project from JBoss incorporating the best features and knowledge from RHQ while at the same time improving on the less strong parts.

Hawkular is composed of a number of individual sub-projects that work together and deliver individual services. The design of those services is in a way that they could also be used standalone in other projects.

The currently most prominent sub-project is Hawkular Metrics, which you probably recall as "RHQ Metrics" and which just has released its version 0.2.7 under the old name. There is also an OpenShift cartridge available for it.

Other sub-projects are:

  • Hawkular-bus: asynchronous bus to connect the various parts. This is a message oriented bus currently running on Active MQ and providing an infrastructure for other projects to re-use.
  • Hawkular-alerts: alerting on incoming metrics (and other events).
  • Hawkular-ui-components: ui components such as Hawt.io 2 plugins and Angular directives that make up the Hawkular Console
  • Hawkular build tools: Helpers and definitions to build Hawkular

Similarly the

  • Wildfly-cassandra extension (run Cassandra 3 as an extension inside WildFly)
  • Wildfly-monitor extension (Monitor WildFly metrics and forward to a Hawkular Metrics instance)

have also been moved over to the Hawkular organization on GitHub

All the pieces are / will be assembled in the Hawkular project

All projects and sub-projects live in GitHub under the Hawkular organization. Right now we have set up the basic projects and other infrastructure. The overall issue tracker is setup at Jboss.org with individual trackers for the sub-projects (you can use the overall tracker and we dispatch).

Right now we are working on getting an end-to-end workflow and integration done for Hawkular.

Development discussions around Hawkular happen on the Hawkular-dev mailing list and you can find the developers hanging around on IRC at irc://irc.freenode.net/#hawkular.

And make sure to follow Hawkular on Twitter

Friday, December 05, 2014

RHQ 4.13 released

I am very pleased to announce the release of RHQ 4.13 as an
early gift from St. Nicholas :)

Screenshot of the RHQ UI showing some charts

This release contains a lot of bug fixes and smaller improvements, as well as some new features:

  • Alerts have a new status 'recovered', that can be filtered upon
  • The UI allows to hide elements that are not needed on a per user basis
  • The as7/WildFly plugin now supports runtime queues and topic subscribers
  • Further improvements in the Storage Nodes

As always RHQ is available for download in form of a zip archive. If you want to try out RHQ without too much setup, you can also use a pre-created Docker image
from https://registry.hub.docker.com/u/rhqproject/rhq-nodb/ (the link contains setup information).

Please consult the release notes for further details and a download link.

Maven artifacts will soon be available on Maven Central.

Special thanks goes to

  • Alan Santos
  • Andreas Veithen
  • Elias Ross
  • Jérémie Lagarde

for their code contributions for this release.

Friday, October 10, 2014

WildFly subsystem for RHQ Metrics

For RHQ-Metrics I have started writing a subsystem for WildFly 8 that is able to collect metrics inside WildFly and then send them at regular intervals (currently every minute) to a RHQ-Metrics server.

The next graph is a visualization with Grafana of the outcome when this sender was running for 1.5 days in a row:

Graphs of JVM memory usage
Graphs of JVM memory usageWildFly memory usage

( It is interesting to see how the JVM is fine tuning its memory requirement over time and using less and less memory for this constant workload ).

The following is a visualization of the setup:


The sender is running as a subsystem inside WildFly and reading metrics from the WildFly management api. The gathered metrics are then pushed via REST to RHQ-Metrics. Of course it is possible to send them to a RHQ-Metrics server that is running on a separate host.

The configuration of the subsystem looks like this:

<subsystem xmlns="urn:org.rhq.metrics:wildflySender:1.0">
<metric name="non-heap"
<metric name="thread-count"

As you see, the path to the DMR resource and the name of the attribute to be monitored as metrics can be given in the configuration.

The implementation is still basic at the moment - you can find the source code in the RHQ-Metrics repository on GitHub. Contributions are very welcome.

Heiko Braun and Harald Pehl are currently working on optimizing the scheduling with individual intervals and possible batching of requests for managed servers in a domain.

Many thanks go to Emmanuel Hugonnet, Kabir Khan and especially Tom Cerar for their help to get me going with writing a subsystem, which was pretty tricky for me. The parsers, the object model and the XML had a big tendency to disagree with each other :-)

Sunday, August 24, 2014

Alert definition templates in plugin(descriptor)s ?


his is a more general question and not tied to a specific version of RHQ (but may become part of a future version of RHQ and/or RHQ-alerts).

Do you feel it would make sense to provide alert templates inside the plugin (descriptor) to allow the plugin writer to pre-define alert definitions for certain resource types / metrics? Plugin-writers know best what alert definitions and conditions make sense and should get the power to pre-define them.

This idea would probably work best with some relative metrics like disk is 90% full as opposed to absolute values that probably depend a lot more on concrete customer scenarios (e.g. heap usage over 64MB may be good for small installations, but not for large ones).
In the future with RHQ-alerts, it should also be possible to compare two metrics with each other, which will allow to say "if metric(disk usage) > 90% of metric(disk capacity) then ...".

I've scribbled the idea down in the Wintermute page on the RHQ wiki.

If you think this is useful, please respond here or on the wiki page. Best is if you could add a specific example.