Archive for the ‘Cloud’ Category

Introducing Microservice Builder

Monday, March 27th, 2017

When the frequency of blog posts drops on this site it generally has two causes: I’m busy and/or I’m working on something that’s IBM Confidential. Both of these have been true over the past six months or so whilst I’ve been working on something we’re calling Microservice Builder. A public beta was announced in the run up to InterConnect which went live on the 24th which means that I can now come up for air and say a little about the work we’ve done so far.

Although not limited to Java deployments, Microservice Builder pulls together multiple strands of work that we’ve been doing in the WebSphere space. First, there is the work that is being done in the MicroProfile community to define a set of standard APIs for building microservices in Java. Initially, this took a set of existing Java EE technologies (JAX-RS, CDI and JSON-P) but now additional APIs are being defined. You can start to see the results of this work in the Liberty March beta where there are new features for injecting environmental configuration and utilizing fault tolerance patterns such as timeout, bulkhead and circuit breaker.

Another area where we’ve sought to improve the developer experience is by providing a fast-path to creating new projects. The Liberty App Accelerator has been around for some time now, allowing you to generate Java projects quickly through a web UI. We’ve taken this idea and extended it to cover Swift and Node.js. This can be achieved either through a web UI or through a new plugin to the Bluemix CLI. (Note that generated projects do not need to be deployed to Bluemix.) The plugin goes beyond just generating projects and allows you to build and run them locally using containers. This means that the developer no longer needs to have the prerequisites (e.g. Java, Maven and Liberty) installed locally.

For a runtime environment, we believe containers are a good fit for microservices and in the first instance we’re focusing on Kubernetes. That could be the newly announced Kubernetes in IBM Containers or it could be on-premises with IBM Spectrum Conductor for Containers. On top of Kubernetes, Microservice Builder adds a lightweight fabric, installed as a Helm chart, that simplifies deployment of Liberty-based services. Specifically, in this first release it generates key and trust stores to facilitate inter-service communication. It also configures an ELK (Elasticsearch-Logstash-Kibana) stack to receive and display information including trace, FFDC, garbage collection and HTTP access logs from the Liberty logstashCollector-1.0 feature.

The final strand of Microservice Builder ties together the development and runtime environments via a Jenkins based pipeline. Once again, this is installed as a Helm chart, and is configured to automatically pick up projects from a GitHub or GitHub Enterprise organization. For a Java application, the pipeline will build and test using Maven, before creating a Docker image and pushing it to a registry. The Docker image is then deployed to a Kubernetes cluster using either the same or a separate pipeline.

To show all of this in action, we have taken the sample conference application from the MicroProfile community and broken it apart in to separate projects to deploy using Microservice Builder. Just follow the docs to recreate it in either your local minikube environment or with Spectrum Conductor for Containers.

Presentations from IBM InterConnect 2017

Sunday, March 26th, 2017

I’m finally back home after what feels like a very long week in Las Vegas at IBM’s InterConnect conference. I promised that I’d post my presentations on SlideShare and I’ll add a few comments here on how each session went.

After an Inner Circle session on Sunday, my first public session of the week was an introduction to containers with WebSphere traditional. This played to a full room which suggests that there is significant interest in the use of containers for existing workloads. Indeed, that was the point of the second half of the session, to describe scenarios where it may make sense to use containers with traditional WebSphere. That’s not to say that it always does and, during one-to-one sessions during the week, I found myself repeatedly cautioning customers against rushing in to the use of containers, particularly with ND, just for the sake of it.

How to Containerize WebSphere Application Server Traditional, and Why You Might Want To from David Currie

My next session covered our new announcement around Microservice Builder. I’ll not say more here as I’ll cover this in a separate post.

Microservice Builder: A Microservice DevOps Pipeline for Rapid Delivery and Promotion from David Currie

Unfortunately, I didn’t get to deliver this session on Liberty and IBM Containers as it clashed with another that I was presenting. As touched on briefly in this presentation, one of the other announcements at the conference was for support for Kubernetes in IBM Containers. There was lots of excitement around this and I urge you to go and check it out for yourself.

WebSphere Liberty and IBM Containers: The Perfect Combination for Java Microservices from David Currie

On Wednesday I had a joint presentation with Brian Paskin looking at options for scalability with Liberty and containers. This was very much Brian’s presentation though so I shan’t post it here. There was an accompanying lab in the afternoon that looked at Liberty collectives and at IBM Containers.

My last session of the week was looking at some of the options when choosing a container orchestration platform: from Liberty collectives, through Swarm and Docker Datacenter, and Kubernetes with IBM Spectrum Conductor for Containers and IBM Containers. Many customers I spoke to this week were looking for a single definitive answer here but my response for now is still very much “it depends”.

Choosing a Container Platform for your WebSphere Applications from David Currie

Find me at IBM InterConnect 2017

Saturday, March 18th, 2017

I’m going to be at IBM’s InterConnect conference this coming week. If you’re going to be there too, there’s a quick run-down of the sessions I’ll be presenting below. The astute will notice that, due to a scheduling snafu, I’m supposed to be presenting two sessions at the same time on Tuesday. If you go to the Liberty/ IBM Containers session then I’m afraid you’ll have to make do with Tom – be kind to him!

If you want to chat about any combination of microservices, containers and WebSphere, you can find me on the microservices ped in the WebSphere area of the expo hall from 5-7:30pm on Tuesday and again from 3-5pm on Wednesday. I’ll be kicking off the latter with a live demo of Microservices Builder, of which more in another post. For Inner Circle customers, I’ll also be talking about this topic at 11am on Sunday.

HAJ-5451 : How to Containerize WebSphere Application Server Traditional, and Why You Might Want To
Date/Time : Mon, 20-Mar, 11:15 AM-12:00 PM
Location : Mandalay Bay South, Level 2 – Surf D
Presenter(s) : David Currie, IBM

BMC-7014 : Roundtable Discussion on Building Java Microservices with WebSphere Liberty
Date/Time : Mon, 20-Mar, 02:00 PM-02:45 PM
Location : Mandalay Bay North, Level 0 – Tropics A
Presenter(s) : Alasdair Nottingham, IBM; David Currie, IBM

BMC-7085 : Meet the Expert on IBM WebSphere Application Server Liberty on Docker
Date/Time : Tue, 21-Mar, 02:30 PM-03:15 PM
Location : Concourse, Bayside B, Level 1 – Meet the Experts Forum # 1
Presenter(s) : David Currie, IBM; Tom Banks, IBM

HAM-5526 : IBM Microservice Builder: A Microservice DevOps Pipeline for Rapid Delivery and Promotion
Date/Time : Tue, 21-Mar, 03:45 PM-04:30 PM
Location : Mandalay Bay North, Level 0 – Islander F
Presenter(s) : David Currie, IBM; Jeremy Hughes, IBM

BMC-5983 : WebSphere Liberty and IBM Containers: The Perfect Combination for Java Microservices
Date/Time : Tue, 21-Mar, 03:45 PM-04:30 PM
Location : Mandalay Bay North, Level 0 – South Pacific A
Presenter(s) : David Currie, IBM; Tom Banks, IBM

BMC-7014 : Roundtable Discussion on Building Java Microservices with WebSphere Liberty
Date/Time : Wed, 22-Mar, 08:00 AM-08:45 AM
Location : Mandalay Bay North, Level 0 – Tropics A
Presenter(s) : Alasdair Nottingham, IBM; David Currie, IBM

BMC-2714 : Utilizing WebSphere Application Server Liberty in Docker Containers for Scalability
Date/Time : Wed, 22-Mar, 10:15 AM-11:00 AM
Location : Mandalay Bay North, Level 0 – South Pacific A
Presenter(s) : Brian S. Paskin, IBM; David Currie, IBM

HAJ-2718 : Utilizing IBM WebSphere Liberty in Docker Containers for Scalability (Lab)
Date/Time : Wed, 22-Mar, 01:00 PM-02:45 PM
Location : Mandalay Bay South, Level 3 – South Seas H
Presenter(s) : Brian S. Paskin, IBM; David Currie, IBM

BAS-5901 : Choosing a Container Platform for Your WebSphere Applications
Date/Time : Thu, 23-Mar, 10:30 AM-11:15 AM
Location : Mandalay Bay North, Level 0 – South Pacific A
Presenter(s) : David Currie, IBM; Tom Banks, IBM

Meetup Happy

Saturday, July 19th, 2014

I’ve gone a bit meetup happy in the past two weeks. Last week I headed along to the Pivotal offices in London for the first London Cloud Foundry User Group meetup organised by one-time colleague Duncan Winn. First to speaker was another ex-Hursley employee, Glyn Normington. He gave a fascinating presentation in to the work that he and his colleagues are doing to replace the backend of Cloud Foundry’s Warden container with libcontainer (now split out from Docker). More on this over on Glyn’s blog.

Next up was London based Tammer Saleh, Director of Products at Pivotal Cloud Foundry Services. You can see the recording of this session from the Cloud Foundry Summit where they talk about the different models for stacking server instances. Finally, James Watters (Vice President of Product, Marketing and Ecosystem for Cloud Foundry at Pivotal) talked about the roadmap for Cloud Foundry in 2014 (including what’s out of scope). See James Bayer’s session from the summit for similar information.

The next meetup was my first at Agile South Coast. If nothing else, this gave me an excuse to have a nose at the new(ish) Ordnance Survey offices! I can’t claim to have been welcomed with open arms to the group (no-one even commented on the fact that they hadn’t seen me there before) but that’s fine by me. Most notable to me though was the fact that I was the only one there who wasn’t a scrum master by profession. Have developers lost interest in agile?

As one would expect with this audience, it wasn’t long before the post-it notes were out and we were collaborating on choosing subjects to discuss. My heart sunk when topics such as “should spikes be given points?” were selected but I was glad when the resounding response from the group seemed to be “it doesn’t really matter – whatever works for you”. Oh, and apparently PSM is more through than CSM but the latter gets more CV points! As I’m part way through reading Kanban in Action, the discussion on Scrum vs Agile in a BAU environment was interesting. I may yet make it to another of these meet ups.

The American style pizza and good selection of beer certainly helped make the trip into town worthwhile although I’ll not mistakenly pick up the 7.2% Sierra Nevada Torpedo Extra IPA in future!

Lastly, I returned to Developer South Coast for a session entitled “NoSQL vs SQL… Fight!”. Actually, there wasn’t much of a fight to be had as the speaker (Tony Rogerson) is an SQL Server DBA. He gave a thorough although halting coverage of the theory behind relational and NoSQL databases though which sadly meant he ran out of time before reaching the potentially more interesting topic of NewSQL databases.

AWS Update

Monday, May 5th, 2014

At the end of April I went to the AWS Summit at ExCeL London, partly to keep up with the competition but largely because the attendees are a different crowd to those you get at your average IBM conference. I managed to miss most of the keynote, partly by design (no early start and an off-peak ticket) and partly due to someone driving in to a level-crossing in Southampton! Having watched the video subsequently, I don’t think I missed a great deal. The only announcements from Amazon that peaked my interest was the arrival of Amazon WorkSpaces in Ireland and the availability of the Twitter stream in Amazon Kinesis.

As in common with the rest of the day, it was the customer slots that were the most interesting. For example, SwiftKey talked about their use of Hadoop on AWS to crunch Wikipedia in other languages to build a starter set for their language models, through to CloudFront as the CDN for serving the final models up to their customers.

I had an interesting chat over lunch with someone who was actually an IBM customer and then wandered the expo watching demos by some of the likely suspects in the cloud deployment, management and monitoring space (Chef, Splunk, DataDog, …).

After lunch the breakout sessions began with six parallel tracks this year. I went to Deployment Done Right first, covering Elastic Beanstalk, OpsWorks and CloudFormation. The only new news for me was an aside that Beanstalk nows supports Docker. It seems like pretty lame support for containerisation though as you appear to get an EC2 instance per image. The accompanying presentation from was most notable for the long list of open source software in use (Nginx, PHP, Magento, Varnish, Redis, Memcached, Elasticsearch, Jenkins, Capistrano, Capify EC2, Boto, …).

Next up was Dynamic Content Acceleration covering the CloudFront CDN and Route 53 DNS with the aim of knocking a second off your response times. The customer this time was which is an interesting site in its own right, providing the capability to turn websites in to structured data (for free).

For the last session of the day I picked Scaling on AWS for the First 10 Million Users which did not, as you might expect, spend a lot of time on auto-scaling, but covered all aspect of application architecture that would contribute to scaling. The customer was the mobile taxi app firm Hailo who are pursuing a micro-services architecture. They are using containerisation (they didn’t specify which) and are apparently writing their own controller to manage the distribution of those containers across EC2 instances to balance workload.

Logsearch & Decker

Friday, April 4th, 2014

Yesterday evening I headed up to the London PaaS User Group meeting as there were two Cloud Foundry related sessions on the agenda. First up was David Laing talking about the open source Logsearch project, a bosh deploy of an Elastic search ELK log analysis cluster. His employer (City Index) has this hooked up to Cloud Foundry system logs and, in some cases, they’re also using it for analysis of application logs with addition parsers. They’re looking for people to get involved in the project and help with the next phase: anomaly detection. One major hole in the solution as it currently stands: it’s only suitable for private PaaS as their is no access control over the logged data.

Up second was an entertaining pitch by Colin Humphreys, Founder and CEO of ours hosts CloudCredo, on how to sell hats to monkeys. That was the back story anyway, it was actually about how there is space in the stack for something that gives you the flexibility of IaaS over what you run but the simplicity of management, scaling and load balancing of PaaS. That something is Container as a Service. Specifically, the ability to push Docker files to Cloud Foundry using a custom stack for the DEA. Something that Colin is referring to as Decker.

Colin gave a nice demo but it is obviously still early days. Currently you can only push Docker files not images. There is also no staging at the moment – the image is created when each instance starts – consequently it is not taking any advantage of intermediate images. There is obviously lots of scope for improvement and it’s definitely one to watch. It was also interesting that Colin is currently focussing on the Docker side with the DEA interactions set to change with the introduction of Diego. The project is open source but Colin recommended waiting until he writes some docs before you try picking it up!