Ted Johnson on Programming & Architecture

"Simplicity is the ultimate sophistication." - Leonardo da Vinci

Wednesday, January 29, 2014

The Technology Architect Organization Quiz - 7 questions to ask before taking the job

The Technology Architect Organization Quiz is a brief set of questions you should ask a potential employer during the interview.  In fact, you should always ask informed questions, after all that is a huge component of an architects job.  Getting a good idea of how a company views, structures, enables architecture is important as there is significant variety.  In some companies architects are looked to as strategic gurus, technology experts, but in others simply quality and design draftsmen, or worse Ivory Tower babblers.  The variety is compounded as there are differences between industries and between Europe and the US.  I have been refining the Technology Architect Organization Quiz for several weeks which and it is in the spirit of The Joel Test by Joel Spolsky.  I highly recommend checking out The Joel Test as it succinctly provides a good measure for development maturity while remaining simple yes or no questions.  I hope the Technology Architect Organization Quiz is just as simple and obvious.  The quiz questions and discussion are more software and application architect focused, but should generally apply to infrastructure, business, enterprise architecture as well.

1. Are architects engaged in strategy and annual planning/budgeting activities?
2. Do developers/engineers within the company want to be architects?
3. How many leaders are between the most senior architect and the CIO/CTO?
4. What is the architect's career path within the company?
5. Does Architecture have regular weekly meetings?
6. Do architects write code, review code, create proof-of-concepts?
7. Is there a defined/followed architecture governance process?
Bonus.  The Joel Test Score?

1.  Are architects engaged in strategy and annual planning/budgeting activities?

A positive answer is critical to your success as an architect within their company.  There are two major reasons architects must be involved in both strategy and annual planning.  First, simply put the longer a project or initiative goes without architecture visibility the more difficult it will be to guide, help, find efficiencies, etc.  Companies often fund what appear to be distinct and separate business initiatives but in fact from a technical perspective have huge overlap, affinity, re-use, interdependencies.  Early plans and strategies can be likened to icebergs.  Icebergs after all remain about 87% below the surface.  Despite only seeing the tip of the iceberg staff structures/workloads and all sorts of hard to change implications can solidify during the strategy and planning meetings.  Architecture must influence before foundation hardens and becomes immovable.  Second, architecture organizations should be providing their own ideas, initiatives, and roadmaps for consideration.  Realistically many great architectural improvements often do not lineup directly with immediate business needs/funding.  Architecture should be submitting proposals and expect to participate early in the process.

2.  Do developers/engineers within the company want to be architects?

Architect roles should be a desirable advancement opportunity for a developer/engineer.  Architects within the organization should be respected, enabled, trusted, and as such have senior positions.  A related question might be: Are many current architects respected past developers?  If developers don't what the job, why not?  It could be a sign that developers don't respect the current architects or that architecture is not well accepted/adopted.  Perhaps the developers and delivery teams have all the real control, feely ignore architecture, or architects are viewed as non-technical overhead.  There could be any number of reasons and all of them are significant warning signs.

3.  How many leaders are between the most senior architect and the CIO/CTO?

Architects typically require a separate chain of command for support, escalation, and engagement.  Also simple measure of value/importance is how connected to the CIO/CTO is architecture?  If all the architects within a company are four levels removed from the executive leader it will almost certainly limit influence, exposure, and budget.  Will delivery focused managers support architects that often have different priorities and potentially conflicting recommendations?  Architectural maturity must be supported, understood, demanded from at the top.  How else can architects be expected to contribute cross-organizational ideas, improvements, paradigm shifts, process changes, and multi-year strategies?  While nothing is impossible and there are great supportive non-architect leaders often there is no substitute for direct architecture path to executive leadership.

4.  What is the architect's career path within the company?

You want to join an architecture organization that is viewed as the 'rock stars' and 'leadership material' within the company.  Who doesn't want to know that there are advancement opportunities?  This can also provide a window into what peers and leaders expect of architects.  It can help you understand if architects are truly consulted/engaged, respected, and valued by peers and leaders.  It is easier to get ahead if architecture is critical to success and engaged on large initiatives.  If architects aren't promoted who is?

5.  Does Architecture have regular weekly meetings?

This question should help you understand if there is an architecture culture at the company.  It is hard to maintain any consistent organizational/practitioner culture if you are not meeting frequently with peer and leader architects.  There may in fact be no real architecture organization, just embedded architects that only take direction from delivery managers.  While this might be fine in some cases, it most certainly would limit architecture success and progression.  How do you get leadership exposure or support combat cost and schedule blinders?  Will your accomplishments only be valued by one delivery leader?  Would such a leader be mature enough to respect that you cannot do your job if you always agree with them?

6.  Do architects write code, review code, create proof-of-concepts?

A healthy architecture organization that is grown and respected from experienced developers and engineers will thus be composed of active programmers/practitioners.  Architects need not always code, but having a sharp relevant (coding) tool set is critical to success.  Architects must be able to understand the systems and technology environment using primary sources, such as code, XML, scripts, etc.  Without practitioner skills architecture must rely on developers.  Moreover a non-technical architect will remain removed from the solution domain and unable to step-in and lead by example, create a proof of concept, learn about the system directly, or even help review quality.  It is not enough for a technical architect (there are other less technical architecture roles) to have not programmed within the last 3 years.

7.  Is there a defined/followed architecture governance process?

In large enterprises the projects need to have scheduled reviews, period.  It should be part of the standard project plan and start-up process.  The format is not as critical who attends and the timing.  The governance meetings/reviews provide critical opportunities to influence, call in leader support, raise awareness, set expectations, and build consensus.  The right mix of technical and execution focused leaders must be present to help drive commitment.   Without these agreed upon check points project teams can let cost and schedule drive decisions requiring more than just an email to adopt a strategic alternative.  These reviews provide checkpoints which reduce last-minute meetings, firefighting, and the feeling that architects are escalating all the time.

Bonus.  How does the company/group score on The Joel Test?

Architects depend on and can be at the mercy of the quality of applications and systems.  Those systems must be maintainable, mature, reasonable to change, and re-useable.  The Joel Test seems like a reasonable way to measure that.  Make sure to account for the type of company and how they use/leverage technology.  The score should be higher for a software development or technology company than a traditional retailer or shipping company.  Just as with other questions a challenge can be an opportunity for an architect/leader given support to make improvements.  However in more junior roles your great ideas and own coding skills might be no match for the chaos and disorder of poorly developed and unmaintainable systems/processes.

How should a company score? 

There is no right score or recommendation.  Obviously, the more correct/positive the better, but a low score might signal a tremendous opportunity.  Maybe they are creating a new architecture function or re-investing in technology staff.  If you are an architect that thrives on challenges and improving processes, standards, etc, a low scoring organization with the right executive support might be a career defining challenge.  Ask these questions or similar ones to set yourself apart from the other candidates.  If they don't like you asking questions then you certainly do not want to work there.

Tuesday, December 17, 2013

Configuring Javascript SyntaxHighligher 3+ in a Blogger site

In this post I tactically outline a quality way to configure the Javascript based SyntaxHighlighter library into a Blogger site. The configuration below makes use of the recent autoloader available in version 3 and greater of the SyntaxHighlighter.  The configuration code below is in fact using the SyntaxHighlighter which incidentally allows you to copy the code examples quickly.  Check out this very popular library here.


1.  Login into blogger.com and click/select the site you want to configure (http://www.blogger.com/home) to support syntax highlighting.  Generally if there are multiple sites you would like to enable each site's template will need to be configured individually.

2.  Click the Template link from the sidebar of the Blogger Site main page shown below.

3.  Save the template, if you choose, by clicking on Backup/Restore in the upper righthand corner and following the instructions.  Better to be safe rather than sorry if you are not familiar with HTML, JS and CSS.  Either way the changes are quick, simple, and typical for those at all familiar with JS and CSS.

4.  Click the Edit HTML button right under the layout which the site currently uses.
Note: We are going to put the Javascript at the bottom of the body and CSS at the top in the header as one always should.
5.  Copy the CSS links into the  ...  tags near the top of the file.  I would recommend copying and pasting this just after the ending tag for the main Blogger CSS
.  See below:
<!-- CSS from CDN JS for SyntaxHighligher -->
<link href='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shCore.css' rel='stylesheet' type='text/css'/>
<link href='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/styles/shThemeDefault.css' rel='stylesheet' type='text/css'/>

Notice a couple of important features.  The files are served up from a solid CDN; this improves performance/reliability.  Additionally the urls allow for either http or https to prevent security warnings which can sometimes happen.

6.  Next copy the Javascript references, again from the CDN, into the bottom just before the
closing tag.  You may already have Javascript there such as google analytics it should not matter if it is before or after.
<!-- SyntaxHighlighter -->
<script src='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shCore.js' type='text/javascript'/>
<script src='//cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shAutoloader.js' type='text/javascript'/>

<script type='text/javascript'>
  'js jscript javascript  //cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJScript.js',
  'xml xhtml xslt html xhtml //cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushXml.js',
  'java  //cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushJava.js',
  'sql  //cdnjs.cloudflare.com/ajax/libs/SyntaxHighlighter/3.0.83/scripts/shBrushSql.js'
SyntaxHighlighter.config.bloggerMode = true;

In the code above you can clearly see how to configure the SyntaxHighligher.autoloader.  This is more efficient as it dynamically loads the addition brushes specified when needed.  This example loads brushes for Javascript, XML/HTML, Java, and SQL.  You can learn more about the autoloader its website: http://alexgorbatchev.com/SyntaxHighlighter/manual/api/autoloader.html.  Additionally, this example includes turning on bloggerMode which seems like an appropriate thing to do.  However, I do not know what it mitigates or provides.

7.  Make sure your tags are closed and then click the Save Template button.  If it saves and Blogger does not report an errors you should be done.  That was not bad at all.    You can now add pretty formatted code to any for your blog posts.

8.  Use SyntaxHighlight in your posts, there is nothing Blogger specific so off you go.  I would recommend referring to the documentation directly: http://alexgorbatchev.com/SyntaxHighlighter/manual/installation.html. It has summarized the pros and cons  of the two approaches well enough.  Also be sure to poke around the additional configuration options as this post just shows the already excellent default.

Monday, May 27, 2013

Here there everywhere...

This site has moved around a couple of times and even lost a little content.  However, I am going to give this another go for a time.  My hope is to make it more of an on-going narrative style to ensure I most more frequently.

Tuesday, November 29, 2011

Amazon Simple Queue Service (SQS) - Simple in good and bad times

Now I am biased having just dealt with IBM's MQ product line and the walled garden approach to their tool suite, so I might be over complementary to Amazon AWS SQS.  Okay, one more thing about IBM MQ, just look at the website, tired, old, selfware; some might argue stable, but hype is the name of the game.  On the other hand AWS SQS fresh and simple, by name, approach, use, and in features.  However, I believe there is something simply excellent about that.

Simple Setup - I had a queue up and running in 30 seconds.  I clicked on the AWS Management Console tab, a browser/java based tool and entered some stuff in and bam queue.  Then I right clicked on it, snappy, I can write a message to it.  So, again I right clicked on the queue entry listed in the AWS SQS console and found a real-time queue reader/listener, and hey I can even delete messages.

Simple Developer Tools - Sure enough I flip over to the eclipse based tools and as expected I could access the queues from there as well.  Nice I can write my 64KB per message max from eclipse, integrate with a plug-in, create a new queue, or whatever.

Simply Service Enable - Most important to me is that every feature is web service enabled.  So, anything Amazon SQS has not automated or simplified can be further refined and utilities written.  To some this may see interesting or novel, but let me assure you that large companies are slowed by walled gardens and management consoles that only the privileged or support companies have access to.  If I dynamically want to create a queue for every batch so I can keep track of them, fine.  If I want a queue for each and every of the million customers, fine, go ahead.  The simple way to put it is that life cycle and direct control matters in large organizations period.  Want to integration just an HTTP call with some XML error handling, which of-course is documented and has a schema, see for yourself, SQS API Documentation.

Simple Documentation - The documentation is short and simple.  I understood how the main drawback or assumption AWS SQS was built on within 5 minutes.  The examples drove every question I had, only so many you can have when you can list the functions on two hands.  Read the architecture document, here.

Simply a message - I can't knock IBM here either, but this is about AWS SQS.  A message can be any text up to 64 KB and can live for up to 14 days.  I have spend weeks trying to wrestle simple non-functional requirements or constraints from product vendors or existing deployed environments.  Not much to question when it is documented and any specific queue can be checked via the management console or a simple service call.

Simply Assumes Idempotent - Which is smartly pants for safe from replay attacks of messages due to outages or glitches.  They chose the word Idempotent and they chose to saddle the consumers of the queue with this to ensure simplified recovery of nodes in their queue clusters.  Now don't get me wrong as designing a consuming system to be Idempotent is not extremely hard, but it pushes the process and data locking onto your application performance.  So you get fast queues, but certain transactions or types of applications will be hard to write.  For example updating a Facebook status or tweeting twice might be fine, or an Order system that commits a transaction, then it is easy for an consuming application to be Idempotent.  However, what about a complex relational multi-transaction rules engine driven process?  Not so easy now.  Now because I cannot trust the AWS SQS to not make a mistake the consuming application could have an architecture performance flaw, which could be avoided by a more rich queue management capability.

Simply distributed and centralized - There is a constant tension in an Enterprise to centralized or disperse.  This is very true for enterprises with countless systems.  AWS SQS is hosted at Amazon, period, and cannot even be easily distributed.  If that data center disappears there could be issues.  If an application wants to use it for internal communication without having to leave the network or memory, AWS SQS will let them down.

Simple Summary - So simplicity is a double edged sword that can provide architectural nirvana or nightmares.  Don't get me wrong I am a huge fan of Amazon's approach to creating service oriented resources for the masses.  They even give you thousands of free messages a year which inspires me to learn more and develop demos with SQS, but I cannot use it for real enterprise demands; darn.

Sunday, January 10, 2010

UML supports the MDD process: No match made in heaven or oxymoron.

This post builds on previous posts (1,2) and is serves as additional commentary and context to a post by  Franco Civello - UML for MDD - Oxymoron or match made in heaven?  Not sure what Model-Driven Development (MDD) is exactly, then check out IBM's MDD Redbook which is one perspective.

UML is a modeling language with thousands of features and facets.  Most people discount UML, even experienced modelers and language developers, without understanding what the language is capable off.  The primary UML feature missed of importance here is the UML Profiles capability.  People miss that UML Profiles allow for extension and constraints of the language itself, read section 18 of the UML Superstructure or take a look at the UML profile catalog.  The two linked posts (1,2) speak to what UML profiles are etc, so you might consider reading them before proceeding.  This extension and constraint mechanism is exactly what is needed to create domain specific models i.e. support MDD.

Now MDD is just a process, so what are the actual implementations or tooling used?  The following are just examples and it is not complete as I am sure there are a hundred ways to do it.  Firstly, there are DSL (Domain Specific Language) tool kits.  A classic example would be YACC and LEX, but a modern example is Eclipse XText Model/Language Definition.  Another significant way is with DSM (Domain Specific Modeling) tool kits, such as Eclipse EMF or MetaEdit+ (a leader in the DSM space).  The DSM space is not free from it's challenges too, but they are further along on average.  Think of DSMs as graphically represented DSLs if you are newer to this area.  Finally UML Profiles are another way to implement the domain specific tooling required.  Tools such as Rational Software Modeler (RSA/RSM), Eclipse MDT, and other tools.

What adds to the confusion between DSL/DSM and UML Profiles when it comes to the creation of domain models for MDD processes is approach and familiarity. The good DSL/DSM tools typically are tabla rasa (blank slate) model creation, much like a empty file when you start programming.  There is less baggage by starting with a blank slate, you also do not have to worry about removing or limiting existing model elements.  Now strictly speaking for UML Profiles you can start with a basically blank slate, but most tools don't support this.  A modeler in the UML Profile space must not only construct/extend the model, but constrain the other model elements that are not needed or conflict with the DSM.  You also have to research and pick the right UML element type to extend to minimize the constraining work you have to do.  Interestingly, in my experience creating small DSMs/logical constructs for a group of architects having a rich language actually reduces the work, but this need not always be the case.

You could really do MDD with any of the tools above, the question often becomes what is the cost, time, and skill level needed by the resources?  What actually seperates the tools are the features and ease of rolling out model tooling to all of the involved parties, such as business, vendors, developers, etc.  So the seven hundred pound gorilla in the room is actually the lack of wide spread usable tooling to build UML Profiles.  Few tools if any implement all of the UML Profile features and creating tooling for use in the MDD process can be very difficult.  I suspect most people use free or drawing oriented UML tools and thus never get exposure to UML Profiles.  Even if they did they would have to buy a specific software suite to support it fulling.  I use Rational Software Modeler (RSM/RSA) which is thousands of dollars and even it does not correctly implement UML Profiles, but it is close.  The DSM tools are not all free, but at least there is Eclipse MDT with EMF/GMF and the generative tools needed.  In a year or two Eclipse may have an offering that is usable enough for novices and support UML Profiles for the tooling efforts more fully.

So UML and MDD go together just fine when you understand the extension, constraining, and profiling capabilities of the language, but all is not well with the union.  UML requires lots of time and energy to learn and understand it before it will begin yielding results.  Better cheap tooling and tutorials are still needed to support the modeling and programming communities at large lest UML/MDD/MDA remain for 'experts only' which I would quantify as a failure by it's advocates.  Much of these challenges are not just limited to a UML Profiles based approach to MDD.

It is worth noting that at some point the industry should look back to Case Tools and YACC/Lex to evaluate why Domain Specific movements have failed in the past, or we may just repeat it.

Monday, January 4, 2010

Reference Cards, Cheat Sheets, Posters, whatever they are called.

Tonight I tried to find quick reference sheets for both UML and BPMN, free of course.  There are quite a fewer different terms for them be it poster, reference cards, cheat sheets, quick guides etc.  However, once I got into several sites by searching for "Reference Cards" I found the other names for these small printable guides to various technologies. Oh yes, and much more managable or likely to be used than "pocket references" which don't fit in your pocket.

List of quality sites quick guides:
itPoster.net (singular)
scribd.com/(kind of)

I did not find a suitable quality UML guide (Stackoverflow has a list), but the BPMN poster at itPoster.net though outdated at time of this post was still really well done. There are several red herring links which get referenced a lot, but are not that good, ex: (How are you to carry this around?) Any other good sites that have a listing of printable reference guides?

Friday, January 1, 2010

Rational & Eclipse UML2 Tools - Flawed UML Component Connections

Connecting UML Components with the Ball and Socket notation in Rational Software Modeler (RSM/RSA) and Eclipse UML2 Tools is not fully implemented or flawed. Too fully understand my point you have to keep in mind that these tools are for modeling, not just drawing a picture/diagram. Thus the internal model representation of what you diagram must be correct and complete.

The Need:
I discovered this issue while trying to design a suitable domain specific language/model for architects to talk about systems and what interfaces/data connections they had between them. The users understand components, so our systems were components. Additionally, interfaces at a high level make sense for being an "agreed upon" contract of interaction between systems. So, the components will realize the interfaces or <> them to show providing and consumption between the systems. The figure below shows two examples of this.

If you look closely at the diagram you will note that the ball and socket do not really connect, which is the first indication that there is a problem. RSM will not let you connect the ball and socket visually or in the model. The Eclipse UML2 Tools will let you connect them, but it fails to model them correctly, though it is still in the early phases of open source development so it kind of gets a pass for most of this discussion.

UML Defines:
So what does UML say should be possible, both from a visual representation (syntax) and model structure (more semantics)? The UML specification 2.2 (Superstructure) says on pg 151 in section 8.1 at a high level that:
"As a result, components and subsystems can be flexibly reused and replaced by
connecting (“wiring”) them together via their provided and required interfaces....
In addition, the BasicComponents package defines specialized connectors for ‘wiring’ components together based on interface compatibility."
The Assembly kind of Connector
At the syntax and semantics level however there are a few details to understand. The major construct to support "wiring" are Connectors. The ComponentBase extends the Connector definition allowing for two kinds of Component specific Connectors, Delegation and Assembly. Also, you "wire" both components and component parts/ports which can make the spec hard to read. It states on page 155 in section 8.3.3:
"The connector concept is extended in the Com ponents package to include interface based constraints and notation.

A delegation connector is a connector that links the external contract of a component (as specified by its ports) to the internal realization of that behavior by the component’s parts. It represents the forwarding of signals (operation requests and events): a signal that arrives at a port that has a delegation connector to a part or to another port will be passed on to that target for handling.

An assembly connector is a connector between two components that defines that one component provides the services that another component requires. An assembly connector is a connector that is defined from a required interface or port to a provided interface or port."
Notice that for assembly it does not say compo nent part, but components and uses the word between. To make things confusing the major examples (pg 153 Figure 8.15) are on Component Instances that are Parts, not between components so to be sure I also found the following on page 156 under constraints:
"[5] An assembly connector must only be defined from a required Interface or Ports to a provided Interface or Port."
An interesting note, but I will go no deeper is that the table 8.2 makes it sound like connecting the ball and socket by a usage/dependency would be equivalent, especially when you factor in
Figure 8.14. Table 8.2 from page 160:
So, the assembly connector defined in UML supports connecting the ball and socket between components, which should be reflected in the model as an Assembly kind of Connector.

Ecore/EMF Provides:
It turns out that the Ecore/EMF model of UML which Rational and Eclipse UML2 Tools leverage support the Assembly Connector per the UML spec. The below two figures show from Eclipse the construction of a component required interface connecting via an Assembly Connector to the same interface provided/realized by a component. The properties could be set and the selector filtered the options correctly given that the interface type had to match.

How Rational (RSM) FAILs gracefully:
In Rational you cannot even create an assembly connector between two components either visually or in the model. The diagram canvas will not allow you to select a component that is not an instance nor will it let you specify it when you are drawing a new (yet to be typed) line. In the model if you try to add a new relationship between two components it will not give you the assembly connector option.

Even worse if you try to connect the ball and socket in Rational with a use or depends you will actually be connecting the interface to itself, see below. Basically the ball and the socket are representations of the interface, not the provided or required ones.

Sadly, Rational provides the Assembly connector and even renders it fine when you are using instances of components (Parts) in the structural view inside a component. So, in my estimation the just mis-understood the specification or they did not keep up with the changes to UML as this was added as part of 2.0 from what I can tell.

A bug will be logged with IBM for the Rational Software Modeler (RSA/RSM) tools, and the ID/Link to it is TDB.