About me • Bosanac Dejan • Senior Software Engineer at FUSESource - http://fusesource.com • Apache ActiveMQ commiter • Co-author of ActiveMQ in Action
Messaging is Loosely coupled exchange of messages between applications Location transparency Can be persistent or non-persistent Can be transactional
Topics One message goes to 0-to-many consumers based on the current subscribers • Think like mailing lists or discussion forums Ideal for publishing business events • Distributed observer pattern • Al ows one part of your system to notify anyone else who may be interested in an event
Queues Messages are load balanced across many consumers • Each message goes to exactly one consumer • Consumers compete for messages Its easy to browse and monitor queues Ideal for grid style applications
What is ActiveMQ?
Apache ActiveMQ Apache ActiveMQ • Leading Open Source messaging platform • Supported Java Standards: o JMS 1.1, J2EE 1.4, JCA 1.5 and XA Reliable, high performance messaging • Out-performs many legacy proprietary message queues • Configurable for many different deployments Multi-Protocol/Multi-Language Support
Background ActiveMQ started in 2005 at CodeHaus Moved to Apache Software Foundation in 2006 914,898 lines of code 24 committers Now the most widely used open source messaging system on the planet
Enterprise Features Failover High Availability Clustering Wide area deployment Management Security
Client Failover Java and C++ clients support seamless failover
High Availability Pure Master/Slave JDBC Master/Slave Shared File System Master/Slave
Pure Master/Slave Shared nothing Ful y replicated • Al messages • Al acknowledgements • Al transactions Slave does not start any transports or network connections
Pure Master/Slave On Master Failure: • Slave wil shutdown • Or Slave will start al network and transports – becoming the master
JDBC Master/Slave Extreme reliability – but not as fast Recommended if already using an enterprise database No restriction on number of slaves Simple configuration
Shared Storage Master/Slave
Shared Storage Master/Slave Recommended if you have a SAN No restriction on number of slaves Simple configuration N.B. – ensure file locking works – and times out – NFSv4 good!
Monitoring Java JConsole – via JMX FuseHQ – via JMX ActiveMQ Web Console Advisory messages Statistics plug-in (agnostic)
Tuning Producers Persistent vs Non-persistent Messages Transactions Embedding Brokers
Non-Persistent Messages Non-persistent delivery is signaficantly faster • Message Producer sends message to the Broker asynchronously • Message is held in temporary storage • 20x improvement in performance
Non-Persistent Reliability Non-persistent may be reliable enough • Messages are cached for resending on the Producer • Broker can filter out duplicates • Consumer can filter out duplicates
Transactions Use Transactions for Persistent Messages • Only the transaction boundary (commit) is synchronous • Messages are batched to storage • Messages cached in broker until a Transaction boundary • Benefits of persistence – performance of non-persistence
Concurrent Consumers Processing messages concurrently – improve performance Message order wil be affected however Can achieve concurrency by: • Multiple Sessions and Consumers • Use JMS ConnectionConsumer • Use a container – e.g. Spring • MessageDrivenBeans
Vertical Scaling Vertical Scaling: Increase load capacity using a single broker can handle. • Enable optimzeDispatch • Disable UseDedicatedTaskRunner • Disable tight encoding in wire format • Increase memory al ocation to the broker • Try using the NIO transport • Upgrade the hardware Cons • There is a limit to the scalability a single machine can give
Horizontal Scaling Horizontal Scaling: Increase load capacity using many brokers • Implemented using a Network of Brokers • Messages a forwarded between brokers to interested consumers. • Lifts the limit of using a single machine to scale. Cons • Advisory messages must be abled to support demand based forwarding. • Delivery failure between brokers will result in re-delivery and a possible duplicate message. • Complex network topologies can lead to non-optimal message routing.
Hybrid Scaling Hybrid of Vertical and Horizontal Scaling • Multiple broker nodes are used by the clients • Brokers are NOT networked • The client application send message to different brokers, typical y based on some defined partitioning of the data. Pros • You can use al the tuning techniques used in Vertical scaling • Have better Horizontal scaleability than using Network Of Brokers (Less broker cross talk) Cons • Added complexity required on the end user Application
Stomp Designed and implemented by ActiveMQ team Designed to be easy to use and implement Lots of client implementations: Ruby,C, Dynamic C, C++, C#, .Net, Delphi, Flash, hx, Perl, PHP, Pike, Python, Smal talk – to name a few.
OpenWire clients Efficient binary implementation Network outage detection Client failover Java, C++ and C# clients.
Web REST Ajax WebSockets
Apache Camel XMPP Gateway FTP Sockets Mail Esper Atom Cometd Many, many more
Delivery Scheduling Modes Timer Cron Hybrid Example message.setStringProperty(ScheduledMessage.AMQ_SCHEDULED_CRON, "30 * * * *"); message.setLongProperty(ScheduledMessage.AMQ_SCHEDULED_DELAY, 10*1000); message.setLongProperty(ScheduledMessage.AMQ_SCHEDULED_PERIOD, 10*1000); message.setIntProperty(ScheduledMessage.AMQ_SCHEDULED_REPEAT, 9); Usage Distribute repeated jobs Replace Cron, Java Timer or Quartz
Future (ActiveMQ 6.0) New Java/Scala Core - fast, scalable, modular Modular design - OSGi friendly, Hot Deployment, Dynamic Configuration New Stores (Apache Cassandra - replicated datacenter store) New Protocols (AMQP 1.0, Beanstalkd, Stomp 1.1 and 2.0) Better Web (RestMQ, WebHooks, WebSockets)
Conclusions Dynamic community Leading in terms of messaging innovation Built for Enterprise Scalable, Good Performance, Reliable