Logo

Who is talking?

Archive

Modern Software Engineering - Part 3 - Designing the organization

4 months ago | Aishwarya Singhal: Aishwarya Singhal

Typical IT organizations have evolved into having multiple layers of managers. Some of that is because organizations try to reduce their risk by having more managers reviewing the work being done. Some is because the growth model only supports growth as managers, and hence everybody grows into a managerial role sooner or later, leading to a pyramid of people that are primarily in supervisory roles. Many organizations have as much as 50% staff in supervisory/ managerial roles. Simply speaking, only 50% of the staff is involved in actual production of software. Basic economics implies that typical overheads (or SG&A) in an organization should be about 20-25%. Shouldn’t the same logic apply to IT teams too? Another aspect here is that complex organization structures lead to a lot of meetings that wastes productive time. At the same time, there is the question of quality being delivered and the trust between different teams. Often, we see a “handover” mindset in most teams - they deliver their part, and then any issues found are to be fixed by the team that comes next in the chain. More often than not, the end-user’s perspective is ignored and forgotten, and teams focus more on covering their backs than on doing the right thing for the user. Let’s look at all these aspects through various enabling mechanisms. Aligned goals and metrics A key aspect of ensuring quality in deliverables is that there is a common definition of quality across the organization. Most teams fail to recognize this, and we see different metrics being used by them. So, while a sales team might be tracking revenue, or customer service team might use Average Handling Time (AHT), the IT team enabling that might still be measuring the number of code releases, or bugs. Now clearly, there is much more than goes into enabling high revenue or low AHT than the software, and there are a lot of IT specific aspects developers need to care for, but that does not mean that the software developers do not have a view on these business metrics. It is vital that everybody uses one language and common metrics across the organization. My most impactful stories have been from situations when my teams took the end-user view and partnered with the stakeholders to ensure that the end result was beautiful. Magic happens when developers and business teams collaborate on achieving common goals. One simple example - we had a feature request to enable printing of VAT invoices for customers, and the developer on my team had already implemented it. However, he did not look happy. I walked up to him to find out why, and I saw him with a printout of an invoice and an envelope. He was upset that the printed customer’s address did not fit in the center of the address cut-out on the envelop. He did not have to do that test, but he went out of his way, fetched an envelope, printed and folded the invoice, and checked if it will work. On the other hand, I was in a team for a large company whose main business was through online sales. Their website crashed, and it had been down for 2-3 days. We were parachuted in as external experts to rescue and fix. At 5 pm, the developers picked their bags and were leaving. We asked the lead developer if he could help debug the issue and he refused - it is the job of the support team and they need to manage it. Now it was late, so I get his point-of-view. However, in such a situation, I would expect an all-hands-on-the-deck type mindset. The disconnect between software developers and business goals is sometimes shocking. Most successful set ups are where all software teams have a business leader who is committed to enabling success and is not just a stakeholder. These business leaders also have sufficient say in the system, typically a direct line to company's leadership. And in such cases every software team is directly responsible for their impact on the business metrics. There will be IT specific metrics that the developers need to track, but they also need to have a keen view on the business goals. I recommend having large screen monitors (that show both business and IT metrics) next to where the developers sit, and I recommend that the teams include the business metrics in their performance reports at least once a month. However, you do not need to over-engineer this. You do not need to track business value or cost per feature. A meta level view is just fine. The goal here is to establish better quality via ownership and awareness, and not to bring in an accounting overhead. Product and platform, not project teams Many organizations work in an outsourcing model even with their internal IT teams. The business team creates a project, gives it to the IT team, and then the IT team has the responsibility to deliver. As expected, this helps optimize the costs (maybe) but erodes quality and trust. The issue here is that most organizations have one model for day-to-day functioning and for mentoring and reporting. This does not have to be. It is important that organizations drop the notion of projects and move towards products. Now “product” has a specific connotation in most organizations - however, we are not talking about the product that you sell to your customer. We are talking about the “software product” that will enable that sale. Although you may sometimes align software product teams with actual products that will be sold to the customer. The difference between a product and a project is that the latter has an end date. It is important that there are product teams that take an end-to-end view on a product, and not a tactical view on enabling a feature/ few features. This enables an improved view on quality and ownership in the teams. This also enables an easier way to align KPIs/ OKRs with the business teams. An easy way to create product teams hence is to follow the business metrics and their responsible business leaders. So, sales may warrant a developer team, customer service might warrant another, and logistics might need yet another. All of them may warrant multiple teams, depending on the number of metrics and business leaders. Another interesting tactic is to allow each business area to have a budget for software development and let them allocate it to each product team based on the latter’s performance in their QBR presentations. This drives collaboration between the business sponsors and the product teams. When you have multiple product teams for a common business area (e.g., sales), you just need all product owners to collaborate with the same business responsible person. Your organization structure does not need to reflect your IT architecture Many IT teams adopt a n-tier architecture, which is composed of different layers. Many of them model their organizations to align with the architecture too - there is a frontend team, a middleware team, a backend team, etc. etc. This leads to a large number of dependencies (and bottlenecks) across teams, and also a lack of end-to-end ownership. In my experience, the most effective model is when organization structure does not replicate the IT architecture. In such cases, there are product teams with end-to-end responsibilities, and platform teams that enable the product teams with tools and frameworks. The platform teams, or as we alternatively call them - IT-for-IT, are deeply technical teams that develop tools and frameworks. Think of these teams as R&D or enabling teams, whose customers are the product teams, and whose primary responsibility is to bring in efficiency and innovation. These are extremely important, and the product owners for these teams need to directly report into the IT leaders. Although we call these platform teams, they should not be centered around specific technical tools, e.g., a Salesforce team, or a SAP team. Salesforce experts, or SAP experts, should be embedded in the right product team. In some cases, the work required is too much to be handled within one “full stack” team. In such cases, there are 2 options, viz., a) take thinner slices of work so a lean team with end-to-end responsibility can still work, or b) divide the teams based on 1-2 layers such that they still have a business significance (e.g., one team does everything until API-enablement, and other prepares frontend and integrates the APIs). The second option is less preferred, and as much as possible, end-to-end ownership should be ensured. More pigs than chicken You need more people that have their skin in the game than those that are just supervisors or advisors. My typical assessment works on the following lines: Anybody who is not actively building or maintaining a product, nor takes an active part in defining the requirements, is an overhead. This includes all advisory roles - security, privacy, architecture, coaches, etc. etc. Anybody spending more that 50% of their time in meetings is an overhead The total number of overhead roles should be less than 25% of the total organization. So, if the IT team is 100 people, at least 75 of them must be actively building the product A simple way to start is to de-layer the organization. A product owner should have a direct reporting to the business leader responsible for that area, and all developers work directly with the product owner and the tech lead, and all tech leads work directly with the IT leader (CIO/ CTO/ VP/ ...). Cut down on all other managerial layers, and clearly define roles and responsibilities for every role Ensure that the Product Owner comes from business team’s perspective and is responsible for writing clear requirements, and for verifying the implementation, and the Tech Lead is a senior developer with >80% time dedicated to coding, and remaining time for mentoring the team. Automate all non-value adding tasks, and simplify what cannot be automated, e.g., coordinator functions, where someone is only responsible to raise a ticket or act as a SPOC for communication. Another example is replacement of manual QA work with automated tests as much as possible. As an example, all advisory roles could be staffed on product teams as needed and would be expected to have an acceptable utilization rate. Typically, such an exercise frees up between 15-20% of capacity that can then be reallocated to value adding roles. The freed-up people are also very talented people in wrong roles, and normally >95% of them can be reallocated (and will be interested) for further value creation. Some might need a bit of training and investing into them brings out magic. Congratulations, you just created a significant productivity boost (through saving and reallocating). At the same time, as a word of caution, do not go overboard with this idea. Many of the advisory teams are often understaffed and underappreciated. In some cases, having SPOCs helps product owners and business leaders to maintain their sanity, especially when it comes to managing vendor relationships. You may still need some manual QA. Similarly, all organizations do require managers, so trying to move towards a near zero managerial capacity will be an absolute disaster. While it is important to chart out an ideal picture, it is also important to then apply a prudent lens and ensure that the model will work in your context. A study at Google indicated that the most effective teams are the ones where team members feel psychological safety and have structure and clarity. I recommend keeping this as the underlying thought when designing the organization. 2 pizza box teams This concept came from Amazon and is almost an industry standard now. The idea is that the team is small enough to have a healthy collaboration and can work together as a SWAT team to deliver towards a common goal. My recipe for typical teams is: 1 Product Owner, 1 Designer, 1 Tech Lead, 4-5 Developers, 1 QA, and 1 Advisor. The designer and advisor role may be fulfilled by different people at different points in time of a product release, based on the need. E.g., there may be a UI designer at 50% and UX designer at 50%, or 50% of architect, 20% of security, and 30% of Subject Matter Experts/ coaches. Some of these may be shared across different teams. So, there are 7-8 dedicated team members, and 2 that are floating. The reason why I would count the floating also into the team is because these need to be in the stand-ups and need to be accountable for the quality of delivery (i.e., they need to be pigs, and not chicken). In special cases, depending on the complexity and (lack of) maturity of the organization, some teams may also have a Business Analyst/ Junior Product Owner, someone that helps the product owner by taking up some of their responsibilities. Functional vs Reporting structures One important clarification to be made here. Everything above talks about how the teams should operate, and not where they should report. The IT team members should continue to report into the IT leaders, so that their career growth, learning and mentoring can be shaped by leaders that understand the field. The product teams should have a dotted line reporting to the business leaders, and the feedback on their performance should be evaluated based on their performance in that context. Another thing to note here is that this does not mean that the IT leaders report into their business counterparts. Both IT and business leaders need to have a top level reporting into the company leadership. This is necessary to ensure that the organization does not always prioritize tactical goals over technical excellence and innovation. This model ensures that the business leaders do not need to worry about the mentorship of technical teams, and the teams get guidance and support from leaders that understand the space. At the same time, the technical teams are focused on generating business value for the organization. Chapters, or communities of practice A final missing piece here is knowledge sharing. It is important that teams share their work for 3 reasons: It enables consistency of implementation across the organization. People have an ability to challenge each other every time they spot an inconsistency. This in turn helps with cost optimizations via prevention of fragmentation and avoidance of duplicate costs It enables learning within a community of similarly skilled colleagues It helps identify training needs for specific skills Spotify has Guilds and Chapters; many other organizations have communities of practice. It is vital to encourage creation of similar virtual structures and ensure that they are exchanging knowledge on a regular basis. So, the community needs to appoint a leader, and that leader should regularly share their observations with the IT leaders. Note that this is not a dedicated role, but an additional responsibility for an existing team member. This has an interesting side-effect: it enables a different growth model in IT compared to traditional ones. Developers can remain developers and still grow (in responsibilities and financial sense) without taking up managerial roles. As always, there is not just one answer for organization structures. Different models work for different set ups, and it is important to understand the context you operate in, and what works in that context. Similarly, the size of an organization can play an important role in defining the feasibility of some of these measures. What works for a 50-member team may not work for a 5000-member organization. Finally, culture and team maturity play an important part in defining the model. At the same time, the principles remain broadly the same, and as long as one can define an execution model that works in their context, it will enable a significant productivity and quality boost in the output. So how do we solve for large organizations? Well, for one, there are a number of standard frameworks and methodologies. I hear SAFe is the most famous. I am personally uncomfortable with any “one-size-fits-all” solutions, so I would recommend evaluating the options based on your context and devising an execution mechanic that works for your organization. Finally, at the heart of all these tips is the intent to simplify (reduce complexity). Anything that increases overheads or complexity in the long term must be challenged and re-evaluated for fit in your context.

Modern Software Engineering - Part 2 - Maximizing developer experience and writing high quality software

4 months ago | Aishwarya Singhal: Aishwarya Singhal

Is the practice of developing software a science (Computer Science), an engineering (software engineering), or an art (software craftsmanship)? When I was in university, we always viewed software as science. We experimented, we learned, and we treated it as mathematics - driven by pure logic. When I started working, it became more of an engineering - applying known techniques, searching for ways how others have solved a problem before, and looking for efficiencies. In recent years, I was introduced to the idea of it being a craft - i.e., focus on quality, and believe in the fact that it can always be improved. I can’t say I fully practice craftsmanship; however, I have moved from engineering more towards it. In my personal view, most projects unfortunately do not quite allow for (or warrant) the time needed for the craft. In any case, we can always do a few things: Ensure a certain level of quality the first time we publish the software through automated checks and thorough code reviews, but avoid over-engineering Make time for refactoring Work smart, not hard - get as many open-source libraries as possible to solve your problems, and only write code for things that are truly specific to your problem and cannot be found on the net Based on these 3 ideas, I have a few practical tips. Build on best-in-class programming techniques My favorite here is the UNIX philosophy that was published in 1978 (yes, over 40 years ago!): * Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features". * Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input. * Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them. * Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them. Why do I love these? These have stayed solid (as has UNIX) over the past 40 years. I derive my coding principles from these, and the following are my most commonly used ones at the moment Write short methods that do one thing only and do it well. This in turn helps to keep a low cyclomatic complexity as well as a smaller number of lines of code per method. I love the Unix pipes and filters, and if you can build that idea into your methods (e.g., using Strategy pattern), a fantastic code quality emerges Use microservices (small pieces of functionality that are independently deployed) where possible Go minimalistic in your design of interfaces, following the YAGNI and Convention over configuration principles. Try to follow the DRY principle as much as possible (without making the code too unreadable) Insist on modularity so that pieces of code can be thrown away when not needed. Caution: avoid over-engineering. This is not the most important aspect if you are following the other principles Refactor, refactor, refactor: Do not shy away from refactoring. The principle is, Whenever you look at a piece of code, aspire to leave it in a better state than you found it in A technique I find useful in writing modular code is: Every time you feel the need to write a comment in the code, see if you can make a new method/ service. Comments usually indicate that the code is doing more than what can be easily understood. You can use any programming language, and any style - my personal favorite at the moment is Functional programming in whichever language I use, because it helps me implement the above-mentioned goals easily. A technique not mentioned here and one I am a big fan of is Event Driven Architecture, (or alternatively, Reactive Programming). It helps reduce dependencies and provides an easier way to guarantee performance and reliability of a system. Align on quality goals and then automate them I have seen situations where the team discussed at length, and kept discussing, the choice of technology. I have also seen similar debates around quality. The only way to avoid an endless debate is to propose and align a set of technologies and quality measures democratically with the team, and then adhere to them for at least a few months. And the best way for that to happen is to automate the agreed principles. Do not define a quality goal that cannot be *(at least partially)* automated, because it is unlikely that it will be implemented. Be real about your Definition of Done (or equivalent) and hold your team accountable to it during code reviews Timebox all decisions Try to leverage industry standards where possible - e.g., Airbnb style for JavaScript linting is often used by teams, or like back in the day Sun’s Java conventions were pretty standard guideline for Java code Call a meeting at the start of the project and agree on quality goals (and publish them) Anybody joining the team afterwards can give suggestions on these goals, but they should only be accepted if they do not disrupt the rest of the team. Alternatively, they can be accepted in the next review of the quality goals (after at least 6-8 weeks) As I mentioned earlier, bugs are any deviation from a user’s expectations. That includes functional defects, and also performance, usability, reliability, etc. Ensure that your quality goals take a complete view. Typical techniques like TDD, Code Reviews, Code Style checks (static code analysis), etc. are usually good measures. When writing automated tests, it is more important to have real quality tests than writing for the sake of getting a 100% test coverage (e.g., you must get a 100% coverage on code containing logic, and it is ok to skip tests for simple Value Objects). Some aspects can be only partially tested - e.g., in case of web accessibility, or security, a manual review may still be required. However, there are many tools available to get you an 80% correct view (if not more) and I would highly encourage using them. Similarly, take your code reviews seriously. GitHub and similar tools simplify the review process significantly and can integrate all feedback that automated tools can generate to help you review code. Technology evolves fast, and I would always recommend checking for the best ways to achieve automation of quality goals before starting any project, and every couple of months even after starting a project. A manual review by the product owner or quality engineer may still be required, but by the time it goes to them, all other checks would have ensured a decent quality level. Be truly agile: ship the software as soon as possible As I said in the previous blog, shipping software is far more important than perfecting it. As long as the code meets all quality goals, it should be good to deploy. I always reflect back on my days in the school, when I first started to code. This was what SDLC looked like to me then: Get a problem (requirements) from the teacher (product owner/ user) Implement them on my computer Copy the working code into a floppy drive (deploy) Show it to the teacher It did not take me weeks or months to do that. It was often done from one day to the next, and in some cases even during a class. Even when we had a project where multiple teammates were working on it, the cycle only had one more step between 3 and 4: Integrate your code with a friend on their computer (aka production set up) There was no 3 or 5 environment set up, no change management, no design approvals, etc. etc. We have made software development overly complex over the years, and it is important to simplify it. The longer you take to ship software, the worse quality you can expect. Now of course, you need to have processes and checks to ensure quality of delivery. However, as long as you have defined sound quality goals and the code meets them all, your code should be good to ship. Put it in front of the customer and address any learnings that come out of that. If you can fix issues quickly, it is perfectly ok to have a few bugs that pop up once you deploy. Some ideas: Use feature branches and feature flags for software development, and have a process to clean up stale feature flags once a feature has been stabilized Ideally, you should push your code to production at least once a day. In the worst case (for complex and large problems), push it within a week. For sub-projects (like a redesign) that takes longer, create a pipeline to deploy the feature branch on the test environment for that sub-project - that’s your production environment for the sub-project. In no case, keep the feature branch alive for more than a week Fully automated deployments: Use continuous delivery pipelines and allow developers to build their own infrastructure through scripts/ bots (infrastructure as code). Achieve a full automation on deployment, ideally including the production environment. In highly controlled settings, implement a fully automatic deployment pipeline at least until pre-production/ staging environment, and then a 1-click deployment for production Ensure sufficient monitoring and logging in the code to observe and learn from user behavior. That will ensure a much higher level of quality than what can be predicted during the development phase and is absolutely needed for a continuous delivery system. CNCF is a great place to start for such topics. Optimize the delivery pipeline to take less than 30 minutes (faster is better) including test execution. This will ensure that developers get feedback on broken builds and issues ASAP and are able to quickly fix the issues on production. One last tip here - be honest to yourself. Every time you have to do a less than perfect job, note down a technical debt item in your product backlog so it is tracked and never forgotten. Reduce the number of meetings you attend One the main time drains for developers is the number of meetings that happen. Avoid them. Put a limit of a total of 30 minutes per day for meetings that need more than 5 people (e.g., the morning standup) for at least 4 days a week. The exception will be some days when you have an architecture/ design session with whole team, planning meeting, or a retrospective, etc. These longer meetings should be on the fifth day of the week. Try to move as much communication online as possible. Use tools like Slack to have effective integrations with various tools and have chats with your team. An online discussion has various advantages - you only dedicate time that you absolutely have to. Also, it helps any other team member to pitch in or learn if they see value in the topic - that makes it much more productive. It is vital to understand the true meaning of agile and I recommend re-reading the Manifesto and listening to the talk Agile is Dead every few months. More often than not, teams claim to work in agile manner but still have numerous complex processes and constraints built around them. Whenever you get an invitation for a meeting, ask yourself - can I avoid this meeting? Try to skip as many meetings as possible. At the same time, pair programming sessions can be awesome. Take a pragmatic view and do those whenever it makes sense. One of the reasons for meetings is dependencies and integrations. Can you reduce them? Try to design your coding responsibilities so you can own end-to-end slices and have minimum dependencies on other teams/ team members. Use interface contracts ) along with techniques like Mechanical Turk, stubs and mocks, to be able to independently develop your code. When done well, this can be done completely independently, and results in significantly reducing integration efforts. Lastly, when I talk about meetings, I am excluding the ones that help you learn (e.g., conferences, meet ups, knowledge exchange sessions). Try to carve out time for them so you do not disrupt your productivity too much, and yet have reasonable time available to learn and share knowledge. As a thumb rule, you should be able to get 6-7 hours a day for focused coding. Leverage and contribute to Open-Source Software, and Internal Open-Source A key aspect to optimizing your time and improving the quality of your code is to leverage open-source libraries as much as possible. Every time you have a problem to solve, check if there is a library that already does that. Ask your team. There is a library for most of the commonly encountered problems - somebody somewhere solved it, stabilized it, and published it. Beware that there are also a number of bad libraries out there, so make sure that a) there is sufficient community behind it, and b) you have tested and seen it working. Open source is awesome because people contribute to it. See what you can publish too. If you solved a generic problem, publish a sanitized library (check for your organization’s policy first). It helps the community of developers, but it also builds a brand for you and your company and attracts good developers to work with you. Similarly, see if you can build an “internal open-source”. If a colleague needs to re-use a piece of code, or if you are re-using code written by someone else, see if it can be a library to be shared internally (or if it generic enough, externally too). Do not greedily create libraries, but instead let that be done on demand. This ensures that a good ecosystem exists for all software in your organization, and everybody benefits from your learnings. At the same time, allow anyone in the organization to submit a pull request, or make changes to the library and help evolve it. That’s the true nature of open-source software and helps with its adoption. Failing this, it just becomes a framework component that will always be your responsibility to maintain and fix and will also see skepticism from your colleagues on its adoption. Finally, find time to learn. Time spent on learning yields exponential results in your productivity (and happiness). Keep measuring quality of your code (through different tools), and you will master it. Happy coding!

Modern Software Engineering - Part 1. Defining a strategy for success

4 months ago | Aishwarya Singhal: Aishwarya Singhal

As leaders, we are often faced with challenges in balancing the needs of the business, and the constraints in delivering to those expectations. It is a complex problem, and one that requires multiple considerations. Over the years, I have developed a list of 5 “principles” I found useful in defining a winning tech strategy. Speed trumps quality, but not always The speed to deliver a software, or a feature, the time to market, is extremely important. At the same time, it is important to focus on the quality. However, these two do not go hand-in-hand. Quality needs time, and that slows down delivery. And nobody likes something that lacks quality even if it is delivered ultra-quick. The definition of acceptable quality changes based on the context. A throw-away software that enables a quick test of a business concept does not need to be perfect. However, a software that is related to hardware which can result in expensive losses due to bugs (like a space shuttle), needs to have a much higher quality level. It is important to know the minimum acceptable levels for both speed and quality. How quickly do you need a feature, and how perfect does it have to be? Usually, in a “Build vs Buy” discussion, “Buy” is more preferable. If you can find an opensource library that already solves a problem, it is better to use it instead of building from scratch. Similarly, a commercial off-the-shelf product may also provide a good foundation and jump start. However, ensure that sufficient due diligence (including a short proof of concept) has been done before adopting/ buying a software. There are a number of horror stories around off-the-shelf products. The marketing material always looks cooler than the actual fit of a software in your ecosystem. In general, an 80-20 rule helps. Ensuring that 80% of scope is delivered with >95% quality is much better than having 100% scope delivered with <80% quality, or only ~20% scope delivered with 100% quality. It is far more important to be able to fix defects quickly, than to avoid them altogether. There will always be unforeseen issues once the software is released to consumers, but if you can fix that in minutes instead of days, nobody notices, and the impact is negligible. An investment into technology - automated delivery (continuous delivery pipelines), monitoring, and processes that enable an on-demand deployment in minutes - will provide a much better risk management ability compared to any review processes that try to foresee and prevent risk. When in doubt, prioritize customer centricity In the B2C world, customer trumps everything else. Even if you are not in B2C, any software you produce needs to be optimized for the consumer. It is extremely important to define metrics and goals with the customers point of view. There are often conflicting priorities, and the engineers would always like to invest into ensuring a robust and maintainable system. As an engineer, I often find myself at the center of this conflict myself. However, as a principle, customer always takes a priority. It is always an unpleasant discussion, but a necessary one. I read a quote from Steve Jobs somewhere: When you’re a carpenter making a beautiful chest of drawers, you’re not going to use a piece of plywood on the back, even though it faces the wall, and nobody will ever see it. You’ll know it’s there, so you’re going to use a beautiful piece of wood on the back. This is a cool quote, and I immensely respect Mr. Jobs, but perhaps this is something that does not apply to most modern software projects. For me, that plywood in the back may be perfectly ok as a way to get started. That does not mean that it should remain there forever. It should be replaced with beautiful wood as soon as possible. But we do not need to wait for the final quality until the software is released, as long as the chest of drawers is usable by the consumer. Does that mean we let poor quality software to be developed? Absolutely not. Optimize for the customer and ensure that only the best quality is presented to them. At the same time, it is important to keep track of “technical debt” - compromises that have been made to urgently ship software to address a business or customer need. And it is important to have a real plan to fix that. Typically, a “technical budget” of 15-20% development capacity is a good way to ensure that the debt does not mount beyond unmanageable levels. Shipping software is far more important than perfecting it A few thoughts to keep in mind here Software sitting on a development or test machine is worthless until it is made available to the consumers The best way to perfect a software is to put it in front of customers and get feedback on it. There is no way one can perfect a software without the customers providing inputs to it The longer you wait to release software, chances are that the quality will be lower. Counter-intuitive? That’s because the longer you wait, the needs of business are likely to evolve. Plus, it will be more complicated to merge all on-going changes being produced by the larger team, and it is more difficult to isolate and fix problems since there is too much change deployed at once I remember working with a colleague who had previously worked in electronics industry - he was stunned that we could modify software and deploy “so quickly”. In hardware world, they had to plan every change, implement the change on a breadboard, send the design to a factory for circuit printing, send the circuits over to the QA department, and work on the feedback. It took them weeks. That’s not how software engineering works though, and it is important to recognize the difference. In today’s world, if a software takes months or years to deliver, somebody’s heart sinks. There are various stories from leading technology companies. Amazon deploys every 11.7 seconds [1], and Google plans for 4 million builds a day [2]. How about the risk of errors due to frequent deployments? Risk management is often misunderstood. In my experience in software engineering, risk mitigation is far more effective than risk avoidance. As long as issues are immediately identified and quickly addressed on production. So, while all change managers will tell you otherwise, set an aspiration for your tech team to deploy multiple times a day, to production. OK - for a greenfield product, you need to first establish a minimum viable product (MVP) on production, before you have multiple deployments a day, but in that case, you only have a production environment once the MVP is ready. It is extremely important to have processes and technology that support multiple daily deployments. I read somewhere: if you are not failing, you are not trying hard enough. Failure is not a problem, not being able to learn or come out of a failure is a problem. Technically, Use Cloud for all deployments - ideally public cloud Automate everything - DevOps, Continuous Delivery, etc. Support zero-touch processes. Anything that requires a human interaction will slow you down Push for an MVP mindset across the board and rationalize the scope for software delivery Shipping software is probably second only to customer centricity in terms of a tech organization’s priorities. Quality is directly proportional to the investment into talent and culture To start with, I am not talking about the financial investment only. I am also talking about time that you invest as a leader. Now, of course, getting quality developers will cost a bit more than the cheapest available in the market. But you do not need the most expensive ones either. Having an all-star team does not guarantee quality. However, a team that sticks together, challenges each other, and believes in the goals of the organization goes a long way in establishing quality. Similarly, the importance of culture cannot be overstated. My key considerations here: Hire quality developers and enable them for success. Let them take decisions. Collective brain power is always better than ivory towers Have a performance centric culture. Celebrate successes and capture learning from failures. However, ensure that people are not scapegoated for failures. The only failures that need to be discussed are where people were comfortable with their status-quo and failed to try or innovate Ensure alignment of common language and goals across the organization. As long as there is a separate “business” and “IT team”, quality will suffer. Ensure that the same goals are used for both, and that they are working as collaborators. Software needs to be business led, and not IT led (although the tech team needs to have a sufficient degree of freedom to bring in tech innovation). Encourage everyone to think of the customer. It is not just the designer’s problem, or the customer service department’s. Spend time with the teams, so they feel connected Invest into the best tooling for the developers. High quality tooling improves productivity, encourages creativity and innovation, and improves people retention. E.g., buying good laptops for developers is a one-time cost, and not a great cost, but significantly improves the quality of their output. Good tooling can also improve collaboration and cut down on unnecessary meetings, which further improves the productivity Ensure that everyone is learning from external community (outside of your company) via meet ups, conferences, or talks delivered by external experts. This needs to happen frequently, and the experts need to be real experts, even if they do not speak the local language Getting quality delivered to customers is hard and it will only happen when the whole organization collaborates, instead of throwing it over the wall to the “IT team”. Be bold: there is no replacement for testing and learning The road to wisdom? — Well, it's plain and simple to express: Err and err and err again but less and less and less. - Piet Hein There is no shortcut to testing. Before the teams start building a product, test the business case. Conduct user tests on cheap prototypes. Not every fancy idea is worth developing, and what may work for another company in another set up, may not always work for you. As a leader, you can (and should) help the teams rationalize their requirements. Once built, measure everything, and capture as much customer feedback as possible. Invest into analytics and capture every customer interaction. Analyze the data for any trends, and feed that back to the technical teams’ “backlog” to be prioritized and implemented. Also, that means that approaches like Mechanical Turk which involves setting up “fake” solutions until “proper” solutions are available, can be fantastic in getting customer insights. The cycle should be: Build -> Measure -> Learn -> Repeat [1]. Shorter this cycle, the better it is. However, a balance is important as always - avoid rabbit holes and know when to pivot. A VC-like mindset is often helpful. So be a coach for the team, encourage testing, but also encourage learning from others and to let go when tests consistently reveal negative results. At the same time, encourage the teams to be bold and bring in innovation from around the world, and not just constrain themselves to a specific sector. Every idea is worth testing. In the end, there is no silver bullet solution, and you will need to review all of these in the context of your organizations. But I certainly hope that these may warrant a discussion within your leadership circles and help define a strategy that works for you.

Modern Software Engineering - Introduction

4 months ago | Aishwarya Singhal: Aishwarya Singhal

Here’s a topic that I have been planning to write about for quite a while now, and I thought a new year is probably a good reason to start penning it down. Software Engineering has naturally evolved since the time the first programs were written. And so have the expectations of its consumers. Today’s world expects everything to be digital. We use our smart phones to read news, to talk to our friends and family, and to perform most of our day-to-day chores. As consumers, we expect good websites, apps and technology enablement from all businesses. (I am going to focus on websites and apps but the same principles can be applied to any software) This expectation has 3 constituents that define our happiness (or our perception of quality): All features we have seen elsewhere must exist (feature parity with competition) It must be easy and quick to use (customer centricity) Everything must work without flaws (bug free software) I have often defined bugs as “deviations in a software’s behavior against stated or unstated expectations”. (Even if no one said they expect a software to work in a certain way, if it does not, they will still be disappointed and will still call it a quality issue). This in turn puts a lot of pressure on businesses and their IT teams (I intentionally draw a difference between the two. We will address it later). The businesses want to deliver to all facets of the customers’ expectations, while managing the cost of delivering them. And the IT teams are flooded with requests, often overwhelmed with conflicting priorities coming from various stakeholders. This makes software engineering much more complex than any other trade - a seemingly impossible scenario. It is only natural then that most IT teams do not deliver to the expectations of their business teams. At the same time, my experience with various large scale IT teams showed a less than 60% time spent by developers on coding features. Even worse, many traditional organizations (businesses for which software is not the core product) have about 50% of team members than are in “overhead” roles - managers, coordinators, etc. - people that are not directly involved in writing the software on a daily basis. So, while there is an ever-increasing expectation of faster delivery, the actual effort spent on delivering the software is about 30-40%. Let’s look at 2 exhibits I found on the internet. [Exhibit 1] [Exhibit 2] The numbers may be different per organization, but we know that the reality is not far off for most of them. So, it naturally begs a question - can we fix this? How do we maximize the software delivery, and cater to our customers’ needs? In this 3-part blog series, I intend to share my perspectives on the various tenets for this topic. We will explore the following: Part 1: Defining a strategy for success Part 2: Maximizing developer experience and writing high quality software Part 3: Designing the organization These blogs will follow in the coming days, and I look forward to hearing your reflections and experiences.

Picking up the pen again

9 months ago | Aishwarya Singhal: Aishwarya Singhal

I haven’t written much in recent years, at least not publicly, and I decided now would be a good time to start again. I also haven’t written much code in recent years, so I decided to re-write Factile in a more popular tech stack. It has been a great experience for various reasons: I could resurrect Factile which has been sitting on a broken server for past 3 years with me having no time to fix it I could experience first hand some of the techs my colleagues have been talking about - its been extremely fulfilling … and it gives me confidence that I can still write a fair quality of code It took me about 4 weeks of few hours a day to completely rewrite it, and I followed the engineering practices I have been coaching my teams on. Factile has been re-written in Node JS and React JS, something that has been a trend for a few years now, and I believe represents a robust developer community. I discarded Scala as a the language for Factile primarily because the frequent churn in the language made it extremely difficult to keep the stack up-to-date and stable in the past, not to mention the fact that it still has a niche in developer community as compared to the ultra vibrant JS community. I chose Cypress for browser testing because a) I wanted to see what the fuss was about, and b) I absolutely love the idea that you can stub API calls inside of tests Finally, I use UptimeRobot for monitoring, CircleCI for continuous integration and APIDoc for, well, API documentation. Of these, APIDoc suprised me the most - it is just amazingly simple to use, and simple to extend (which I did in a way because I did not like the default template/ color schemes). I have been using CircleCI since 2015 and I think recently, it has become incredibly complex with poor documentation. I could have used TravisCI I guess, but I’ll stay with Circle for now. Oh yes, I didn’t use Typescript. Why? I personally don’t like my JS code to have a need to compile, and I feel that instead of writing types for JS, I would rather write code in Scala (or Java, or Kotlin) ;-)

Write a Java program to read console data using bufferedreader API

about 1 year ago | Subodh Gupta: Subodh's Blog

BufferedReader is a class which simplifies reading text from a character input stream. It buffers the characters in order to enable efficient reading of text data. The buffer size may be specified, or the default size may be used. The default is large enough for most purposes. In general, each read request made of a Reader causes a corresponding read request to be made of the underlying character or byte stream. It is therefore advisable to wrap a BufferedReader around any Reader whose read() operations may be costly, such as FileReaders and InputStreamReaders. For example, // BufferedReader in = new BufferedReader(new FileReader("foo.in")); will buffer the input from the specified file. Without buffering, each invocation of read() or readLine() could cause bytes to be read from the file, converted into characters, and then returned, which can be very inefficient.import java.io.BufferedReader;import java.io.IOException;import java.io.InputStreamReader; public class BufferedReaderExample {  public static void main(String[] s) throws IOException {    // Wrap System.in with InputStreamReader followed by BufferedReader    BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(System.in));    System.out.print("Enter Name: ");      // Finally read from bufferedReader     String name = bufferedReader.readLine();    System.out.println("Name: " + name);  }}

Absorbing Feedback

over 1 year ago | Subodh Gupta: Subodh's Blog

We all want to improve, absorbing feedback is the most important step in the process of improvement. Even when we get feedback, we don't know how to act on it.We have good techniques for giving feedback like SBI (Situation, Behavior, Impact) however, it took me some time to develop a good framework to absorb and act on feedback. Let me start with a real-life situation:A couple of years back, I received feedback, however incapable of acting on it. I was struggling to make any improvement. During this phase, I developed the framework which helped me. This framework has two attributes "Understand Deeply" and "Act Swiftly". Understand Deeply is about understanding the feedback and situations where behaviour is depicted so that you can relate to the situation. Some situations bring out the same undesired behaviour. It is also very important to understand the intent of the feedback. Act Swiftly talks about acting on feedback, taking steps to implement the feedback and correct the behaviour. It is really important to act promptly on feedback, people are more likely to give feedback if they see action on it.These behaviours don't work in isolation but rather has an amplifier impact when combined together.Let's start with the first quadrant on the top left corner and go anti-clockwise.Insincere quadrant, in this quadrant feedback, is well understood however not ready to act. There could be multiple reasons for not acting on a feedback example it requires some fundamental behaviour change, requires too much effort etc. One way to change is to try to divide the feedback into smaller pieces and tackle a piece at a time. Indifferent quadrant this is the hardest quarter to be in where you don't (want to) understand the feedback because the intent to act on the feedback is missing. Change needed in this situation is fundamental. Burnout, de-motivation or lack of interest could be some of the reasons. Taking a break sometimes help, if its a team, work or manager related issue changing the team or even the company might also help.Next comes Frustrated quadrant where feedback is being acted upon is not getting addressed and all the is going waste. It's best to take a step back to try and understand the feedback more deeply. Same feedback in a different situation could have very different meanings.True and lasting change happens when we understand the feedback and act upon it promptly to go through change-learn-improve cycles. Remember when the feedback is well understood it becomes easy to incrementally improve.Hope this helps.

Using Result Combinator Functions in Rust

over 1 year ago | Pat Shaughnessy: Pat Shaughnessy

Rust’s Result type can help you control your program’s flow by checking for errors in a succinct, elegant way Using Rust for the first time, error handling was my biggest stumbling block. Was this v

How Rust Makes Error Handling Part of the Language

over 1 year ago | Pat Shaughnessy: Pat Shaughnessy

In Spanish these are all “dedos,” while in Englishwe can distinguish between fingers and toes. Learning a foreign language can be an incredible experience, not only because you can talk to new people, v

Using Rust to Build a Blog Site

over 1 year ago | Pat Shaughnessy: Pat Shaughnessy

Rust comes with batteries included (source: Wikimedia Commons) After “Hello World,” blog sites a

Hello blog reader

over 1 year ago | Pranav Modi: pranavmodi.com

Hello there! Advertisements

Frameworks and Why (Clojure) Programmers Need Them

over 1 year ago | Daniel Higginbotham: Flying Machine Studios

It seems like there's a strong aversion to using frameworks in the Clojure community. Other languages might need frameworks, but not ours! Libraries all the way, baby! This attitude did not develop without reason. Many of us came to Clojure after getting burned on magical frameworks like Rails, where we ended up spending an inordinate amount of time coming up with hacks for the framework's shortcomings. Another "problem" is that Clojure tools like Luminus and the top-rate web dev libraries it bundles provide such a productive experience that frameworks seem superfluous. Be that as it may, I'm going to make the case for why the community's dominant view of frameworks needs revision. Frameworks are useful. To convince you, I'll start by explaining what a framework is. I have yet to read a definition of framework that satisfies me, and I think some of the hate directed at them stems from a lack of clarity about what exactly they are. Are they just glorified libraries? Do they have to be magical? Is there some law decreeing that they have to be more trouble than they're worth? All this and more shall be revealed. I think the utility of frameworks will become evident by describing the purpose they serve and how they achieve that purpose. The description will also clarify what makes a good framework and explain why some frameworks end up hurting us. My hope is that you'll find this discussion interesting and satisfying, and that it will give you a new, useful perspective not just on frameworks but on programming in general. Even if you still don't want to use a framework after you finish reading, I hope you'll have a better understanding of the problems frameworks are meant to solve and that this will help you design applications better. Frameworks have second-order benefits, and I'll cover those too. They make it possible for an ecosystem of reusable components to exist. They make programming fun. They make it easier for beginners to make stuff. Last, I'll cover some ways that I think Clojure is uniquely suited to creating kick-ass frameworks. (By the way: I've written this post because I'm building a Clojure framework! So yeah this is totally my Jedi mind trick to prime you to use my framework. The framework's not released yet, but I've used it to build Grateful Place, a community for people who are into cultivating gratitude, compassion, generosity, and other positive practices. Just as learning Clojure makes you a better programmer, learning to approach each day with compassion, curiosity, kindness, and gratitude will make you a more joyful person. If you want to brighten your day and mine, please join!) What is a Framework? A framework is a set of libraries that: Manages the complexity of coordinating the resources needed to write an application... by providing abstractions for those resources... and systems for communicating between those resources... within an environment... so that programmers can focus on writing the business logic that's specific to their product I'll elaborate on each of these points using examples from Rails and from the ultimate framework: the operating system. You might wonder, how is an OS a framework? When you look at the list of framework responsibilities, you'll notice that the OS handles all of them, and it handles them exceedingly well. Briefly: an OS provides virtual abstractions for hardware resources so that programmers don't have to focus on the details of, say, pushing bytes onto some particular disk or managing CPU scheduling. It also provides the conventions of a hierarchical filesystem with an addressing system consisting of names separated by forward slashes, and these conventions provide one way for resources to communicate with each other (Process A can write to /foo/bar while Process B reads from it) - if every programmer came up with her own bespoke addressing system, it would be a disaster. The OS handles this for us so we can focus on application-specific tasks. Because operating systems are such successful frameworks we'll look at a few of their features in some detail so that we can get a better understanding of what good framework design looks like. Coordinating Resources Resources are the "materials" used by programs to do their work, and can be divided into four categories: storage, computation, communication, and interfaces. Examples of storage include files, databases, and caches. Computation examples include processes, threads, actors, background jobs, and core.async processes. For communication there are HTTP requests, message queues, and event buses. Interfaces typically include keyboard and mouse, plus screens and the systems used to display stuff on them: gui toolkits, browsers and the DOM, etc. Specialized resources are built on top of more general-purpose resources. (Some refer to these specialized resources as services or components.) We start with hardware and build virtual resources on top. With storage, the OS starts with disks and memory and creates the filesystem as a virtual storage resource on top. Databases like Postgres use the filesystem to create another virtual storage resource to handle use cases not met by the filesystem. Datomic uses other databases like Cassandra or DynamoDB as its storage layer. Browsers create their own virtual environments and introduce new resources like local storage and cookies. For computation, the OS introduces processes and threads as virtual resources representing and organizing program execution. Erlang creates an environment with a process model that's dramatically different from the underlying OS's. Same deal with Clojure's core.async, which introduces the communicating sequential processes computation model. It's a virtual model defined by Clojure macros, "compiled" to core clojure, then compiled to JVM bytecode (or JavaScript!), which then has to be executed by operating system processes. Interfaces follow the same pattern: on the visual display side, the OS paints to monitors, applications paint to their own virtual canvas, browsers are applications which introduce their own resources (the DOM and <canvas>), and React introduces a virtual DOM. Emacs is an operating system on top of the operating system, and it provides windows and frames. Resources manage their own entities: in a database, entities could include tables, rows, triggers, and sequences. Filesystem entities include directories and files. A GUI manages windows, menu bars, and other components. (I realize that this description of resource is not the kind of airtight, axiomatic, comprehensive description that programmers like. One shortcoming is that the boundary between resource and application is pretty thin: Postgres is an application in its own right, but from the perspective of a Rails app it's a resource. Still, hopefully my use of resource is clear enough that you nevertheless understand what the f I'm talking about when I talk about resources.) Coordinating these resources is inherently complex. Hell, coordinating anything is complex. I still remember the first time I got smacked in the face with a baseball in little league thanks to a lack of coordination. There was also a time period where I, as a child, took tae kwon do classes and frequently ended up sitting with my back against the wall with my eyes closed in pain because a) my mom for some reason refused to buy me an athletic cup and b) I did not possess the coordination to otherwise protect myself during sparring. When building a product, you have to decide how to create, validate, secure, and dispose of resource entities; how to convey entities from one resource to another; and how to deal with issues like timing (race conditions) and failure handling that arise whenever resources interact, all without getting hit in the face. Rails, for instance, was designed to coordinate browsers, HTTP servers, and databases. It had to convey user input to a database, and also retrieve and render database records for display by the user interface, via HTTP requests and responses. There is no obvious or objectively correct way to coordinate these resources. In Rails, HTTP requests would get dispatched to a Controller, which was responsible for interacting with a database and making data available to a View, which would render HTML that could be sent back to the browser. You don't have to coordinate web app resources using the Model/View/Controller (MVC) approach Rails uses, but you do have to coordinate these resources somehow. These decisions involve making tradeoffs and imposing constraints to achieve a balance of extensibility (creating a system generic enough for new resources to participate) and power (allowing the system to fully exploit the unique features of a specific resource). This is a very difficult task even for experienced developers, and the choices you make could have negative repercussions that aren't apparent until you're heavily invested in them. With Rails, for instance, ActiveRecord (AR) provided a good generic abstraction for databases, but early on it was very easy to produce extremely inefficient SQL, and sometimes very difficult to produce efficient SQL. You'd often have to hand-write SQL, eliminating some of the benefits of using AR in the first place. For complete beginners, the task of making these tradeoffs is impossible because doing so requires experience. Beginners won't even know that it's necessary to make these decisions. At the same time, more experienced developers would prefer to spend their time and energy solving more important problems. Frameworks make these decisions for us, allowing us to focus on business logic, and they do so by introducing communication systems and abstractions. Resource Abstractions Our software interacts with resources via their abstractions. I think of abstractions as: the data structures used to represent a resource the set of messages that a resource responds to the mechanisms the resource uses to call your application's code (Abstraction might be a terrible word to use here. Every developer over three years old has their own definition, and if mine doesn't correspond to yours just cut me a little slack and run with it :) Rails exposes a database resource that your application code interacts with via the ActiveRecord abstraction. Tables correspond to classes, and rows to objects of that class. This a choice with tradeoffs - rows could have been represented as Ruby hashes (a primitive akin to a JSON object), which might have made them more portable while making it more difficult to concisely express database operations like save and destroy. The abstraction also responds to find, create, update, and destroy. It calls your application's code via lifecycle callback methods like before_validation. Frameworks add value by identifying these lifecycles and providing interfaces for them when they're absent from the underlying resource. You already know this, but it bears saying: abstractions let us code at a higher level. Framework abstractions handle the concerns that are specific to resource management, letting us focus on building products. Designed well, they enable loose coupling. Nothing exemplifies this better than the massively successful file abstraction that the UNIX framework introduced. We're going to look at in detail because it embodies design wisdom that can help us understand what makes a good framework. The core file functions are open, read, write, and close. Files are represented as sequential streams of bytes, which is just as much a choice as ActiveRecord's choice to use Ruby objects. Within processes, open files are represented as file descriptors, which are usually a small integer. The open function takes a path and returns a file descriptor, and read, write, and close take a file descriptor as an argument to do their work. Now here's the amazing magical kicker: file doesn't have to mean file on disk. Just as Rails implements the ActiveRecord abstraction for MySQL and Postgres, the OS implements the file abstraction for pipes, terminals, and other resources, meaning that your programs can write to them using the same system calls as you'd use to write files to disk - indeed, from your program's standpoint, all it knows is that it's writing to a file; it doesn't know that the "file" that a file descriptor refers to might actually be a pipe. Exercise for the reader: write a couple paragraphs explaining precisely the design choices that enable this degree of loose coupling. How can these choices help us in evaluating and designing frameworks? This design is a huge part of UNIX's famed simplicity. It's what lets us run this in a shell: # list files in the current directory and perform a word count on the output ls | wc The shell interprets this by launching an ls process. Normally, when a process is launched it creates three file descriptors (which, remember, represent open files): 0 for STDIN, 1 for STDOUT, and 2 for STDERR, and the shell sets each file descriptor to refer to your terminal (terminals can be files!! what!?!?). Your shell sees the pipe, |, and sets ls's STDOUT to the pipe's STDIN, and the pipe's STDOUT to wc's STDIN. The pipe links processes' file descriptors, while the processes get to read and write "files" without having to know what's actually on the other end. No joke, every time I think of this I get a little excited tingle at the base of my spine because I am a: This is why file I/O is referred to as the universal I/O model. I'll have more to say about this in the next section, but I share it here to illustrate how much more powerful your programming environment can be if you find the right abstractions. The file I/O model still dominates decades after its introduction, making our lives easier without our even having to understand how it actually works. The canonical first exercise any beginner programmer performs is to write a program that prints out, Wassup, homies?. This program makes use of the file model, but the beginner doesn't have to even know that such a thing exists. This is what a good framework does. A well-designed framework lets you easily get started building simple applications, without preventing you building more complicated and useful ones as you learn more. One final point about abstractions: they provide mechanisms for calling your application's code. We saw this a bit earlier with ActiveRecord's lifecycle methods. Frameworks will usually provide the overall structure for how an application should interact with its environment, defining sets of events that you write custom handlers for. With ActiveRecord lifecycles, the structure of before_create, create, after_create is predetermined, but you can define what happens at each step. This pattern is called inversion of control, and many developers consider it a key feature of frameworks. With *nix operating systems, you could say that in C programs the main function is a kind of onStart callback. The OS calls main, and main tells the OS what instructions should be run. However, the OS controls when instructions are actually executed because the OS is in charge of scheduling. It's a kind of inversion of control, right? 🤔 Communication Frameworks coordinate resources, and (it's almost a tautology to say this) coordination requires communication. Communication is hard. Frameworks make it easier by translating the disparate "languages" spoken by resources into one or more common languages that are easy to understand and efficient, while also ensuring extensibility and composability. Frameworks also do some of the work of ensuring resilience. This usually entails: Establishing naming and addressing conventions Establishing conventions for how to structure content Introducing communication brokers Handling communication failures (the database is down! that file doesn't exist!) One example many people are familiar with is the HTTP stack, a "language" used to communicate between browser and server resources: HTTP structures content (request headers and request body as text) TCP handles communication failures IP handles addressing Conventions The file model is a "common language", and the OS uses device drivers to translate between between the file model and whatever local language is spoken by hardware devices. It has naming and addressing conventions, letting you specify files on the filesystem using character strings separated by slashes that it translates to an internal inode (a data structure that stores file and directory details, like ownership and permissions). We're so used to this that it's easy to forget it's a convention; *nix systems could have been designed so that you had to refer to files using a number or a UUID. The file descriptors I described in the last section are also a convention. Another convention the file model introduces is to structure content as byte streams, as opposed to bit streams, character streams, or xml documents. However, bytes are usually too low-level, so the OS includes a suite of command line tools that introduce the further convention of structuring bytes by interpreting them as characters (sed, awk, grep, and friends). More recently, more tools have been introduced that interpret text as YAML or JSON. The Clojure world has further tools to interpret JSON as transit. My YAML tools can't do jack with your JSON files, but because these formats are all expressed in terms of lower-level formats, the lower-level tools can still work with them. Structure affects composability. The file model's simplicity is what allows it to be the "universal I/O model." I mean, just imagine if all Linux processes had to communicate with XML instead of byte streams! Hoo boy, what a crazy world that would be. Having a simple, universal communication system makes it extremely easy for new resources to participate without having to be directly aware of each other. It allows us to easily compose command line tools. It allows one program to write to a log while another reads from it. In other words, it enables loose coupling and all the attendant benefits. Communication Brokers Globally addressable communication brokers (like the filesystem, or Kafka queues, or databases) are essential to enabling composable systems. Global means that every resource can access it. Addressable means that the broker maintains identifiers for entities independently of its clients, and it's possible for clients to specify entities using those identifiers. Communication broker means that the system's purpose is to convey data from one resource to another, and it has well-defined semantics: a queue has FIFO semantics, the file system has update-in-place semantics, etc. If Linux had no filesystem and processes were only allowed to communicate via pipes, it would be a nightmare. Indirect communication is more flexible than direct communication. It supports decoupling over time, in that reads and writes don't have to happen synchronously. It also allows participants to drop in and out of the communication system independently of each other. (By the way, I can't think of the name for this concept or some better way to express it, and would love feedback here.) I think this is the trickiest part of framework design. At the beginning of the article I mentioned that developers might end up hacking around a framework's constraints, and I think the main constraint is often the absence of a communication broker. The framework's designers introduce new resources and abstractions, but the only way to compose them is through direct communication, and sometimes that direct communication is handled magically. (I seem to recall that Rails worked with this way, with tight coupling between Controller and Views and a lack of options for conveying Controller data to other parts of the system). If someone wants to introduce new abstractions, they have to untangle all the magic and hook deep into the framework's internals, using -- or even patching! -- code that's meant to be private. I remember running into this with Rails back when MongoDB was released; the document database resource was sufficiently different from the relational database resource that it was pretty much impossible for MongoDB to take part in the ActiveRecord abstraction, and it was also very difficult to introduce a new data store abstraction that would play well with the rest of the Rails ecosystem. For a more current example, a frontend framework might identify the form as a resource, and create a nice abstraction for it that handles things like validation and the submission lifecycle. If the form abstraction is written in a framework that has no communication broker (like a global state container), then it will be very difficult to meet the common use case of using a form to filter rows in a table because there's no way for the code that renders table data to access the form inputs' values. You might come up with some hack like defining handlers for exporting the form's state, but doing this on an ad-hoc basis results in confusing and brittle code. By contrast, the presence of a communication broker can make life much easier. In the Clojure world, the React frameworks re-frame and om.next have embraced global state atoms, a kind of communication broker similar to the filesystem (atoms are an in-memory storage mechanism). They also both have well defined communication protocols. I'm not very familiar with Redux but I've heard tell that it also has embraced a global, central state container. If you create a form abstraction using re-frame, it's possible to track its state in a global state atom. It's further possible to establish a naming convention for forms, making it easier for other participants to look up the form's data and react to it. (Spoiler alert: the framework I've been working on does this!) Communication systems are fundamental. Without them, it's difficult to build anything but the simplest applications. By providing communication systems, frameworks relieve much of the cognitive burden of building a program. By establishing communication standards, frameworks make it possible for developers to create composable tools, tools that benefit everybody who uses that framework. Standards make infrastructure possible, and infrastructure enables productivity. In this section I focused primarily on the file model because it's been so successful and I think we can learn a lot from it. Other models include event buses and message queues. I'm not going to write about these because I'm not made of words, ok?!? Environments Frameworks are built to coordinate resources within a particular environment. When we talk about desktop apps, web apps, single page apps, and mobile apps, we're talking about different environments. From the developer's perspective, environments are distinguished by the resources that are available, while from the user's perspective different environments entail different usage patterns and expectations about distribution, availability, licensing, and payment. As technology advances, new resources become available (the Internet! databases! smart phones! powerful browsers! AWS!), new environments evolve to combine those resources, and frameworks are created to target those environments. This is why we talk about mobile frameworks and desktop frameworks and the like. One of the reasons I stopped using Rails was because it was a web application framework, but I wanted to build single page applications. At the time (around 2012?), I was learning to use Angular and wanted to deploy applications that used it, but it didn't really fit with Rails's design. And that's OK. Some people write programs for Linux, some people write for macOS, some people still write for Windows for some reason (just kidding! don't kill me!). A framework is a tool, and tools are built for a specific purpose. If you're trying to achieve a purpose the tool isn't built for, use a different tool. More Benefits of Using Frameworks So far I've mostly discussed how frameworks bring benefits to the individual developer. In this section I'll explain how frameworks benefit communities, how they make programming fun, and (perhaps most importantly) how they are a great boon for beginners. First, to recap, a framework is a set of libraries that: Manages the complexity of coordinating the resources needed to write an application By providing abstractions for those resources And systems for communicating between those resources Within an environment So that programmers can focus on writing the business logic that's specific to their product This alone lifts a huge burden off of developers. In case I haven't said it enough, this kind of work is hard, and if you had to do it every time you wanted to make an application it would be frustrating an exhausting. Actually, let me rephrase that: I have had to do this work, and it is frustrating and exhausting. It's why Rails was such a godsend when I first encountered it in 2005. Frameworks Bring Community Benefits Clear abstractions and communication systems allow people to share modules, plugins, or whatever you want to call framework extensions, creating a vibrant ecosystem of reusable components. If you accept my assertion that an operating system is a framework, then you can consider any program which communicates via one of the OS's communication systems (sockets, the file model, etc) to be an extension of the framework. Postgres is a framework extension that adds an RDBMS resource. statsd is an extension that adds a monitoring resource. Similarly, Rails makes it possible for developers to identify specialized resources and extend the framework to easily support them. One of the most popular and powerful is Devise, which coordinates Rails resources to introduce a new user authentication resource. Just as using Postgres is usually preferable to rolling your own database, using Devise is usually preferable to rolling your own authentication system. Would it be possible to create a Devise for Clojure? I don't think so. Devise is designed to be database agnostic, but because Clojure doesn't really have a go-to framework that anoints or introduces a go-to database abstraction, no one can write the equivalent of Devise in such a way that it could easily target any RDBMS. Without a framework, it's unlikely that someone will be able to write a full-featured authentication solution that you can reuse, and if you write one it's unlikely others would see much benefit if you shared it. I think it's too bad that Clojure is missing out on these kinds of ecosystem benefits. Another subtler benefit frameworks bring is that they present a coherent story for how developers can build applications in your language, and that makes your language more attractive. Building an application means coordinating resources for the environment you're targeting (desktop, mobile, SPA, whatever). If your language has no frameworks for a target environment, then learning or using the language is much riskier. There's a much higher barrier to building products: not only does a dev have to learn the language's syntax and paradigms, she has to figure out how to perform the complex task of abstracting and coordinating resources using the language's paradigms. If your goal is to create a mass-market product, choosing a language that doesn't have frameworks for your target environments is a risky choice. Finally, frameworks become a base layer that you can create tooling for. The introduction of the filesystem made it possible for people to write tools that easily create and manipulate files. Rails's abstractions made it easy to generate code for creating a new database table, along with an entire stack - model, view, controller - for interacting with it. Frameworks Make Development Fun If you still think frameworks are overkill or more trouble than they're worth, believe me I get it. When I switched from Rails to Clojure and its "libraries not frameworks" approach, I loved it. A framework felt unnecessary because all the pieces were so simple that it was trivial for me to glue them together myself. Also, it was just plain fun to solve a problem I was familiar with because it helped me learn the language. Well, call me a jaded millenial fart, but I don't think that this work is fun anymore. I want to build products, not build the infrastructure for building products. I want a plugin that will handle the reset password process for me. I want an admin panel that I can get working in five minutes. Frameworks handle the kind of work that ideally only has to be done once. I don't want to have to do this work over and over every time I want to make something. For me, programming is a creative endeavor. I love making dumb things and putting them in front of people to see what will happen. Rails let me build (now defunct) sites like phobiatopia.com, where users could share what they're afraid of. The site would use their IP address to come up with some geo coordinates and use Google Maps to display a global fear map. A lot of people were afraid of bears. Frameworks let you focus on the fun parts of building an app. They let you release an idea, however dumb, more quickly. Frameworks Help Beginners Frameworks help beginners by empowering them to build real, honest-to-god running applications that they can show to their friends and even make money with, without having to fully understand or even be aware of all the technology they're using. Being able to conjure up a complete creation, no matter how small or ill-made, is the very breath of wonder and delight. (I don't know exactly what this means, but I like how it sounds!) There's a kind of thinking that says frameworks are bad because they allow beginners to make stuff without having to know how it all works. ActiveRecord is corrupting the youth, allowing them to build apps without even knowing how to pronounce SQL. There's another line of thinking that says it's bad to try to make things easier for beginners. It's somehow virtuous for people to struggle or suffer for the sake of learning. Hogwash. Fiddlefaddle. Poppycock. Joy beats suffering every time, and making learning more joyful allows more people to reap the benefits of whatever tool or product you've created. I am a photographer. I have a professional camera, and I know how to use it. Some of my photos require a fair amount of technical knowledge and specialized equipment: tea This isn't something you can create with a camera phone, yet somehow I'm able to enjoy myself and my art without complaining that point-and-shoot cameras exist and that people like them. Novices benefit greatly from expert guidance. I don't think you can become a master photographer using your phone's camera, but with the phone's "guidance" you can take some damn good photos and be proud of them. And if you do want to become a master, that kind of positive feedback and sense of accomplishment will give you the motivation to stick with it and learn the hard stuff. Frameworks provide this guidance by creating a safe path around all the quicksand and pit traps that you can stumble into when creating an app. Frameworks help beginners. This is a feature, not a bug. A Clojure Framework Frameworks are all about managing the complexity of coordinating resources. Well, guess what: Managing Complexity is Clojure's middle name. Clojure "Managing Complexity" McCarthy-Lisp. Personally, I want a single-page app (SPA) framework, and there are many aspects of Clojure's design and philosophy that I think will make it possible to create one that seriously kicks ass. I'll give just a few examples. First, consider how Linux tools like sed and awk are text-oriented. Developers can add additional structure to text by formatting it as JSON or YAML, and those text-processing tools can still work the structured text. In the same way, Clojure's emphasis on simple data structures means that we can create specialized structures to represent forms and ajax request, and tools to process those structures. If we define those structures in terms of maps and vectors, though, we'll still be able to use a vast ecosystem of functions for working with those simpler structures. In other words, creating specialized structures does not preclude us from using the tools built for simpler structures, and this isn't the case for many other languages. Second, Clojure's abstraction mechanisms (protocols and multimethods) are extremely flexible, making it easy for us to implement abstractions for new resources as they become available. Third, you can use the same language for the frontend and backend!!! Not only that, Transit allows the two to effortlessly communicate. This eliminates an entire class of coordination problems that frameworks in other languages have to contend with. In my opinion, the Clojurian stance that frameworks are more trouble than they're worth is completely backwards: Clojure gives us the foundation to build a completely kick-ass framework! One that's simple and easy. One can dream, right? My ambition in building a SPA framework is to empower current and future Clojure devs to get our ideas into production fast. I want us to be able to spend more time on the hard stuff, the fun stuff, the interesting stuff. And I want us to be able to easily ship with confidence. The framework I'm building is built on top of some truly amazing libraries, primarily Integrant, re-frame, and Liberator. Integrant introduces a component abstraction and handles the start/stop lifecycle of an application. re-frame provides a filesystem and communication broker for the frontend. Liberator introduces a standard model for handling HTTP requests. If my framework is useful at all it's because the creators of those tools have done all the heavy lifting. My framework introduces more resources and abstractions specific to creating single-page apps. For example, it creates an abstraction for wrapping AJAX requests so that you can easily display activity indicators when a request is active. It creates a form abstraction that handles all the plumbing of handling input changes and dispatching form submission, as well the entire form lifecycle of fresh, dirty, submitted, invalid, succeeded, etc. It imposes some conventions for organizing data. As I mentioned, the framework is not quite ready for public consumption yet becaause there's still a lot of churn while I work out ideas, and because there's basically no documentation, but I hope to release it in the near future. If you'd like to see a production app that uses the framework, however, I invite you to check out Grateful Place, a community site for people who want to support each other in growing resilience, peace, and joy by practicing compassion, gratitude, generosity, and other positive values. By joining, you're not just helping yourself, you're helping others by letting them know that you support them and share their values. Please click around and look at the snazzy loading animations. And if you feel so moved, please do join! I love getting to interact with people in that context of mutual support for shared values. One of the only things I care about more than Clojure is helping people develop the tools to navigate this crazy-ass world :D In the mean time, I'll keep working on getting this framework ready for public consumption. Expect another blawg article sharing some details on how Grateful Place is implemented. Then, eventually, hopefully, an actual announcement for the framework itself :) If you don't want to wait for my slow butt, then check out some ofthe amazing Clojure tools that already exist: Luminus Fulcro which probably does everything I want my framework to, only better re-frame remains my favorite frontend framework duct is great but its docs aren't that great yet Coast on Clojure, a full stack web framework (Sorry if I neglected your amazing Clojure tool!) Thanks to the following people who read drafts of this article and helped me develop it: Mark Bastian Dmitri Sotnikov aka @yogthos Joe Jackson Sergey Shvets Kenneth Kalmer Sean whose last name I don't know Tom Brooke Patrick whose last name I don't know (update: It's Patrick French!) Fed Reggiardo Vincent Raerek Ernesto de Feria Bobby Towers Chris Oakman The TriClojure meetup

Verify your website accessibility

almost 2 years ago | Sumit Bajaj: Sumit Bajaj's Blogs

Web accessibility is the practice of ensuring that your website is interactive and accessible to even people with disabilities. Many countries provide laws protecting the rights of disabled persons, therefore it becomes really important to design and develop the website in compliance with those laws and follow web accessibility as de-facto.The blog helps to test your web accessibility.1. Open chrome browser and browse any website. Open developer tools and click on Audit tab. Scroll to Audits section and check the 'Accessibility' option.Web accessibility check in Chrome browser2. Once you click on 'Run audits' button, you will see a message(as shown below). Don't worry, auditing has started.Starting auditing of website3. Once audit is completed, it will generate audit report(as shown below)Audit report for web accessibilityHave a great day !!

Buddy Systems at Work: a Framework for Rewarding Relationships on Remote Teams

almost 2 years ago | Daniel Higginbotham: Flying Machine Studios

One of my work mottos is "support the hell out of each other": helping your friends grow and succeed is far more rewarding than chasing after the next gold star for yourself. While I was at Reify Health I was lucky enough to work with other devs who had a similar ethos. There were no rock stars, and everyone looked out for each other. I'm happy to have gotten to work somewhere where I made real, honest-to-god friends :) During my final weeks there I collaborated with coworkers on an experimental framework for supporting each other in a formal context. This framework would let us explore how to be there for each other in more meaningful ways while maintaining boundaries. This seemed particularly valuable in Reify's mostly-remote environment. With their permission, I'm sharing the initial framework proposal below: The Buddy System This doc outlines a proposal for a buddy system. The main idea behind the buddy system is for us to support each other by regularly spending time chatting with each other about things beyond whatever story is at hand. Exactly what that looks like will be up to you; this doc includes suggestions for how to have rewarding conversations. First, here are some ground rules: Ground Rules It’s confidential. Whatever you talk about remains between you and your buddy unless you say otherwise. This is meant to provide each other with time and space to speak about what’s going on :D It’s completely voluntary. You don’t have to participate, and if you do participate you can shape it however you want to: the frequency and content of chats is up to you. You can opt out or change buddies whenever you want. If you decide to drop out of the buddy system, it’s totally up to you if you want to explain why, or not. The buddy system is meant to consciously cultivate a space where we can be vulnerable with each other; part of making that safe is setting the expectation that people can opt out without having to explain why. By the same token, in joining the buddy system you accept that your buddy may at some point wish to opt or change buddies, and you commit to try to not take that personally if it happens 👍 Every three months we’ll change buddies - this gives us a chance to grow together, and also a chance to keep getting new perspectives by buddying with other folks on the team. There’s no impact on your yearly review. Though “Buddy System” would be a wonderfully Orwellian name for a mandatory chatting initiative. It’s reciprocal. Buddies commit to listening to and sharing with each other. If you feel like you have room to grow with listening or sharing, that’s totally cool - and a good first step is to share that with your buddy :) It’s not hierarchical. Being a Buddy means whole-hearted listening, asking questions, and just being supportive, which is independent of any junior or senior designations. The point is to support each other as human beings, not to meet project objectives :-) Suggestions Tell your buddy what you want to get out of the buddy system. Do you want to talk about your career? Your goals at Reify? How to improve our architecture? By letting your buddy know what you’re looking for, you can get all Jerry Maguire and help them help you. Listen your asses off. Avoid the urge to give advice unless asked for it, but instead try to truly understand what your buddy is saying, and try to help them gain clarity. Potential topics include goal-setting and “life at Reify.” Goal setting: It can be very useful to talk to someone about your broader goals and to have them hold you accountable. Sharing a goal often makes it more meaningful and gives you more motivation to reach it. Life at Reify: We have retros, which are useful, but regular one-on-one conversations also have a place in supporting each other These topic suggestions are not meant to be exhaustive. When asking for feedback, communicate what kind of feedback you want: A pat on the back High-level feedback Nitty-gritty feedback Notes Daniel is acting as facilitator in that he’s organizing discussion about the Buddy System so that we all come to a shared understanding of what it is and what we want to get out of it. However, Daniel isn’t, like, the Buddy System’s owner, and can’t tell you what to do. Some reading this might be uncomfortable with the ideas outlined in the buddy system - and that's totally ok! A key part of the framework is you don't have to buy into it. But I think it speaks volumes about the level of trust and mutual good feeling among the team that almost everyone was excited about it while respecting that we're all different and not everyone prefers the same way of interacting. On the other hand, new hires were especially excited because they felt like it would help them acclimate to their new environment. To me, a team that is enthusiastic about these ideas is a team that I'd want to work with; it's a team that actually likes the idea of relating to each other on a real, human level. One topic that came up was, how is this different from just being friends? Is it really necessary to have a structure that everyone agrees to? Does that bring an element of artifice into friendship? My thinking is that structure is a precondition for vulnerability. Structure tells us to be extra mindful in how we interact with each other, and it provides boundaries (like a clear time limit) to protect us from getting overwhelmed. By designating a time and place to consciously interact with each other, it also resolves the problem of never knowing when is "the right time" to bring something up. This would be helpful for new hires, sure, but even in my personal life I've found that structure is much better at fostering deeper interactions than just winging it. Knowing that everyone is "in it" together and that "it" won't last forever provides a solid foundation for trust. Final note: vulnerability here does not mean you have to spill your guts, emotionally leaking on each other all the time. It does mean feeling safe to express that you're struggling or that you feel dissatisfied or hurt in some way. What are your thoughts? Have you tried something like this where you worked, and how did it go? P.S. Reify is hiring!

Grow faster by making yourself redundant!

almost 2 years ago | Subodh Gupta: Subodh's Blog

Redundant! but why? What about job security? Shouldn't we be consolidating our position instead of going redundant? All these questions are obvious when you read the title of this article. Don't worry, we are not talking about how to get yourself fired but opposite, how to grow faster by making yourself redundant in a role. In a job, we are expected to play roles that are needed from us e.g. program manager, project manager, people manager, technical lead are few roles engineering managers play to be successful. This article talks about how we can progressively outgrow in our current role and start learning (or mastering) the new roles. Transferring responsibilities to people you are managing are the best way of making space for learning something new. Over the past couple of years as an engineering manager, I have been following this philosophy to learn new things. Let's take an example of leading a project to understand this better.Engineering Managers are the backbone of the team. However, they are also the weakest link in the team, often they are the single point of failure. The team rely on them to make crucial decisions. In a normal situation, nothing is wrong with this arrangement but when they are away (in conferences, multi-day-meetings, holidays or sick) projects gets derailed due to decrease in decision making velocity and lack of process ownership in the team.If EM spends too much time on project execution, they won't have enough time for people (call it 100% harvesting and no investing). In my experience, time spent on people is directly proportional to the happiness of the team which has a flywheel effect on deliverables. On the other hand, unhappy teams lead to lower development velocity and error-prone deliverables, which results in managers spending more time on execution.This cycle leads to burnout in managers and also results in unhappy teams which have a domino effect on deliverables (poor in both quantity and quality). To avoid this situation the most important step for the managers is to automate the execution and funnel more-time in people management and product planning. However, it's not a trivial task and involves rolling out well-orchestrated process changes in small implement-and-improve iterations. For me, it took 6 months to achieve a sustainable execution model.This article will take you through the journey of "making myself redundant".Phase 1: "Doing all the things an EM should be doing! Well, I want to change it!" Identifying delegation worthy responsibilities! Question: What are the responsibilities my team can own and run?Planning? Retrospective? They do it fortnightly hence have a very good understanding which means they can be independent on this quite quickly.Question: Who should own it? Do we need a new role? What should be the responsibilities of this new role?Probably we need a feature/project lead. Who should be a Lead?A-Team member.Why don't I ask my team, what they think they should be doing as a Lead?Time for brainstorming: We as a team gathered in a room and listed responsibilities which will be owned by the lead. E.g. updating the project status, finalizing implementation approach, sprint planning,retro, demos (team rotates these) etc. Point worth noting, we didn't define everything, like how to manage external dependencies or how to distribute work. The plan was to start with something small and keep on increasing the responsibilities. Why not define comprehensive roles and responsibilities? Defining clear R&R in one go is very easy and leads to very clear paths however it also means decoupled roles which are recipe for disaster when you starting something this big. We can compare it with athlete training, they can't run a marathon in the first week but the distance is increased gradually over a sustainable duration. Similarly, we decided on some base responsibilities with adequate checks in place e.g. leads can always reassign part or whole of responsibility to me (EM) at any moment after discussion.Is this a good model?There are multiple ways a lead can be trained onto new responsibilities:Coach a lead on everything before handing over the responsibility "Throw them in water and let them learn how-to-swim" give them responsibility and let them ask for help.Start small take a step at a time, keep on adding responsibilities. I personally like the third option as it leads to sustainable handover where everyone gets the time to adjust and iterate.Phase 2: "Attending all the team meetings as a participant."Feature lead is starting to onboard on new responsibilities and taking ownership more and more meetings. However, it will start eating into the capacity of the lead which could be a problem. So, apart from defining some basic responsibilities, we also finalized on a rule "lead should start with 20% time on leading". The idea here is to increase the responsibility and not to make them unproductive.The right mix!As discussed above, listing role and responsibilities help, however it can also be over-prescriptive. Agreeing on "What needs to be done", "When it needs to be done" in important whereas "How" can be left open.Ask pointed questions, to make sure things are on track, it will surface the trade-offs and flaws.  Follow five Whys approach if and needed.Phase 3: "Coaching the lead!"As lead is taking responsibilities, start spending more time with them to make sure they understand why we are doing something instead of just doing it for the sake of it. This will lead conversations on "why this and not that", addressing these questions is like laying the foundation of the leadership, answering every question sincerely will help in developing a good leader for your team.A healthy team is built on healthy processes"Processes are the building blocks of trust in a team". "Teams who learn to rely on each-other can solve any problem". "If we can define a problem, we can solve it and if we can solve it once, we should solve it efficiently the next time".Healthy team need lightweight (healthy) processes to make sure everything runs smoothly. You can rely on the team and processes to take care of everything.Phase 4:"Give up the power, not the control!"The last piece of the puzzle is patience.Following meetings if well run can help a lot:PlanningStand-upRetrospectivesHealth-monitors SWOTIf teams is not happy and motivated you can't deliver anything worthwhile. Listening to the team's concerns is the easiest way to address the problems. If team is raising a concern it should be fixed, don't apply filters without understanding the problem first. Team will always be motivated if you remove (or help them remove) the impediments, remember motivation is key to the successful delivery. I believe the most important part of my job is to keep the team motivated. I get the real sense of how we are doing as a team from the meetings listed above.Right Now!Phase 5: "Close to doing nothing related to a project execution! \o/Focus on long-term strategy and career planning for the team!"We successfully delivered the project without delay, covering more than we anticipated and removing all the major blockers we faced.As an EM, I need to be do something to keep my anxiety in check.So, I am making sure:Scope is fixed Team stays away from distractions Focus on one thing at a time, minimize context switching Taking discussions irrelevant to the team away from them Address action items on priorityGiving away your responsibilities is not easy and people would be making mistakes like you did when you started. However, how to make this learning rewarding is the challenge. Remember you are learning as much as the other person, you are learning to be patience, you are learning to give away control, you are learning to diagnose problems early, so much learning is involved in this process. These are the building blocks of being a leader, if you can't delegate you can't scale.In the end, learning is an essential part of the growth for everyone. Focus on growing people in your team for the unforeseen results. Encouraging leadership in your team is the best way of growing your team.Try being redundant, its addictive!

Creating and Installing Sitecore Packages

over 2 years ago | Sumit Bajaj: Sumit Bajaj's Blogs

Creating and Installng Sitecore PackageCreate Sitecore Package1. Open Sitecore(from where you would like to create package) and click on Package Designer2. Once Package designer panel is opened, click on Item statically and provide all required details as shown below3. After clicking on 'Item statically' option, a popup will open. Choose selected items and click on 'Add with SubItems' or 'Add Items' buttons    a. clicking on 'Add Item' will only include selected item and not child items    b. clicking on 'Add with subitems' will include selected Item and its child items4. Click on 'Generate Zip' option which will generate the package in .zip format5. Download the .zip file of sitecore package by clicking on download buttonInstalling Sitecore Package1. Open Sitecore (instance where you would like to install package) and click on 'Installation Wizard'2. upload the .zip package to install and click next to install3. During install, Sitecore popup will appear for inputsa. Override - This option replace the entire subtree with the subtree in the package. You have to be very careful while selecting Override option as it may delete all child items if you have not added the Item with child items while creating package. b. Merge - This is safe option which will find the matching items and versions with those from the package but do not replace any subitemsPlease feel free to post your comments in case you see any concerns or like to thank you :).

Summer School With The Rust Compiler

over 2 years ago | Pat Shaughnessy: Pat Shaughnessy

(source: Steve Klabnik via Twitter) A few months ago, I saw this tweet from Steve. I'm not even sure what "derridean

Quick Tutorial - Setup a Solr instance

over 2 years ago | Sumit Bajaj: Sumit Bajaj's Blogs

Before we start installing Solr instance on any machine, we should be clear on few concepts.What is Solr?Solr is Apache's product and a fast open-source Java search server.Why we need it?Solr enables you to easily create search engine for websites, databases and files.How can we install it?You can download any version of Solr from http://archive.apache.org/dist/lucene/solr/. Steps to install after it is downloaded-Extract the downloaded Solr package and copy it on C or D drivePlease make sure latest version of Java is installed on machine. You can start Solr instance by below command: java -jar start.jarBy default, Solr will run on port 8983. However you can change it ../etc/Jetty.xml file. Look for <set name="port"><SystemProperty name="jetty.port" default="8983"></set> and change it.You can keep the below command in .bet file and run it automatically. "C:\solr-6.1.0\bin\solr" restart -h localhost -p 8984Please feel free to send email for any help.

From ActiveRecord to Diesel

almost 3 years ago | Pat Shaughnessy: Pat Shaughnessy

Learning something new every day is exercise for your mind. (source: Jeremy Bowcock via Wikimedi

Could not load file or assembly 'Microsoft.Web.Infrastructure'

over 3 years ago | Sumit Bajaj: Sumit Bajaj's Blogs

Could not load file or assembly 'Microsoft.Web.Infrastructure, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.What 'Micorosoft.Web.Infrastructure' does?This dll lets HTTP modules register at run time.Solution to above problem:Copy 'Micorosoft.Web.Infrastructure' dll in bin folder of your project and this problem should be resolved. If you have .Net framework installed on machine, this dll should be present on it. You can search for this dll and copy it in your active project folder.  Alternatively, you can install this dll using nuget package managerPM> Install-Package Microsoft.Web.Infrastructure -Version 1.0.0Happy coding!!

Learning Rust: If Let vs. Match

over 3 years ago | Pat Shaughnessy: Pat Shaughnessy

Human languages have similar words with different shades of meaning. Some computer languages do too. (from: Wikimedia Commons

The Death of Microservice Madness in 2018

over 3 years ago | Dave Kerr: dwmkerr.com

There are cases where great efforts have been made to adopt microservice patterns without understanding the specifics of the problem at hand...

Effective Shell Part 3: Getting Help

over 3 years ago | Dave Kerr: dwmkerr.com

In this article I'll show you how to quickly get help when working with tools in the shell, without disrupting your flow!

Looking Inside Postgres at a GiST Index

over 3 years ago | Pat Shaughnessy: Pat Shaughnessy

What do Postgres GiST indexes look like? How are they similar or different from standard Postgres indexes? In the last few posts in this series (http://patshaughnessy.net/2017/12/11/trying-to-represe

Saving a Tree in Postgres Using LTREE

over 3 years ago | Pat Shaughnessy: Pat Shaughnessy

In my last post, I showed you how to install and enable a Postgres extension called https://www.postgresql.org/docs/current/stati

Installing the Postgres LTREE Extension

over 3 years ago | Pat Shaughnessy: Pat Shaughnessy

Hidden inside of your Postgres server is code that provides special SQL operators and functions designed to support tree operations. It’s called the LTREE exte

Trying to Represent a Tree Structure Using Postgres

over 3 years ago | Pat Shaughnessy: Pat Shaughnessy

Suppose you had a hierarchical data structure in your application - how would you save it in a database? How would you represent a complex tree structure using flat rows and columns? There are a few different, equally valid

Integrating OpenShift and Splunk for Docker Container Logging

over 3 years ago | Dave Kerr: dwmkerr.com

In this article I'm going to show you how to set up OpenShift to integrate with Splunk for logging in a Docker container orchestration environment. These techniques could easily be adapted for a standard Kubernetes installation as well! Screenshot: Counter service splunk The techniques used in this article are based on the Kubernetes Logging

Effective Shell Part 2: Become a Clipboard Gymnast

over 3 years ago | Dave Kerr: dwmkerr.com

In this article I'll show you how you can use the shell as an efficient tool to compliment how you use the clipboard.

Effective Shell Part 1: Navigating the Command Line

almost 4 years ago | Dave Kerr: dwmkerr.com

Text in the command line can quickly get unwieldy. Here are some simple tricks you can use to navigate in the command line.

Promises, Async and Await in Node.js

almost 4 years ago | Rocky Jaiswal: Still Learning

This week we had the much awaited Node.js 8.0.0 release! Node is getting faster and better all the time. This release also makes the async / await feature available natively. But before we look at async / await let us quickl ...

A utility to help you wait for ports to open

almost 4 years ago | Dave Kerr: dwmkerr.com

There are occasions where you might need to have scripts or commands which wait for TCP/IP ports to open before you continue. I've come across this need again and again when working with microservices, to make my life easier I've created a little utility called wait-port which will wait

Engineering Leader: How to make a difference?

almost 4 years ago | Subodh Gupta: Subodh's Blog

Being an engineering leader is not an easy task, especially when you are stepping into this role as a first-timer. Seldom you go through a training, however in most situations you are expected to figure it out yourself with little or no guidance. Often leadership demands you to play following roles:People ManagerProduct/Business knowledgeArchitect/Senior DeveloperVision for technologyI will be writing separate blogs to cover the first two roles. For a beginner, last two roles are very important to gain credibility of the team which is the most important factor in succeeding at your job.As a technical leader, you will be facing the challenges from all the directions like decision making, improving team efficiency and choosing the technology roadmap for your team. However, I hope adopting following patterns can help you sail through these challenges.Decision Making: When hit with the problem, as a techie you rely on your technical skills and often start suggesting solutions immediately which is fine in few situations, however in my experience better way would be to ask pointed questions to lead your team in driving the right decision. Questions should be repeatedly asked to highlight the blemishes in the suggested design. For example, "How would you take care of security?", "How are you addressing scalability?", "How would application behave under these circumstances?" and many more.It's easier to jump to the solution by picking up the first suggestion by trusting the loudest (not literally) person in the room. However, as a coach, you should be involving everyone in the decision-making process. Find out what is missing in the room, is it the knowledge, the skill or the participation. You need to create enough opportunities for everyone to participate. To do that you need to think about various factors like structure of the team, personalities in the team etc. These factors can be neutralized to an extent by bringing everyone to a common understanding of the problem. In my experience, following preparation before hosting an important meeting would make it easier:Share a document explaining the problem which can educate the team before the discussionDuring the meeting, explain the problem in detail so that team can align their understandingDo your homework before going to the meeting. This will help you in asking the right questions and guiding the meeting towards the solution. Improving Efficiency: Team delivers to the full potential if they are not disturbed and allowed to focus on the problem. As a leader, think about how the team is getting distracted, how can you make them focus on stories or task in hand? Improve the development processes so that you can minimize the impact on the team. For example, in a production issue churn out the non-technical part and direct developer towards solving the technical problem.Vision for Technology: Successful teams stay ahead of the technology curve. As a coach, you need to ensure that your team is adapting to changing landscape of the technology world. While working with your team to strengthen the technology muscle, you need to start focusing on what should be the future technology platform for your team. If you are not thinking at least 3-months down the line, you will lose the war while you were busy fighting battles.Be sensitive to what role you are playing in your team. You are not a developer anymore, you are a coach and coaching means understanding before suggesting, listening before speaking, advising rather than commenting.As a beginner, starting in the world of unknown unknowns of management, I hope my secret sauce can help you prepare the right dish for your team!!!

Techniques for Efficiently Learning Programming Languages

about 4 years ago | Daniel Higginbotham: Flying Machine Studios

Learning programming languages is a skill: do it well and you'll experience one dopamine hit after another as you master something new; do it poorly and you'll feel constantly frustrated and even give up. What follows are the best techniques for learning programming languages that I've picked up over years of teaching programming by writing books and articles, doing talks, and running a training course. Many of these techniques are pulled from books explaining the latest research in efficient learning, and you can find those books (along with other great programming books) at Community Picks: Learn Programming. Test Yourself Constantly to Defeat The Illusion of Competence One of the worst ways to learn is to re-read or re-watch material. This kind of review gives you the feeling that you understand the topic covered because it seems like you're understanding the topic effortlessly. Researchers call this the illusion of competence. A significantly better approach (and one of the best techniques you can employ) is to test yourself constantly. Instead of re-reading what a function or class or object is, ask yourself to define these concepts or use them in a short program; force yourself to somehow demonstrate your understanding. This process often feels uncomfortable, but it's much more efficient at forming long term memories. You can take this one step further and test yourself before you've covered the material by, for example, attempting exercises before reading a chapter. Remarkably, this has also been shown aid memory formation. The impressive impact that testing has on learning is called the testing effect, and here are some specific ways you can take advantage of it: Before reading a chapter or watching a video, try guessing at what you're about to learn and write it down. Try doing a chapter's exercises before reading the chapter. Always do exercises, even the hard ones. It's OK to give up on an exercise and come back to it later (or never, even), but at least try it. (More on this in the next section.) Read a short program and try to recreate it without looking at the original code. Or, go smaller and do this with a function. Immediately after learning a new concept like objects, classes, methods, or higher-order functions, write code that demonstrates that concept. Create diagrams that illustrate concepts, both in isolation and how they relate to each other. Blog about a concept you just learned. Try explaining the concept to a non-technical friend. (I did this a lot when writing Clojure for the Brave and True; being able to explain an idea in layman's terms forces you to understand the idea deeply.) Many of these techniques boil down to write some code! With programming it's easy to believe we're learning a lot just by reading because programming is text-heavy and conceptual. But it's also a skill, and like any other skill you have to perform it to get better. Writing code is the best way to reveal your incorrect assumptions about programming. The faster you do that, the faster you can make corrections and improve. If you'd like to learn more about the testing effect, check out make it stick: The Science of Successful Learning. Take Time to Unfocus If you're stuck on a problem or don't understand something you just read, try taking a walk or even a shower -- anything to enter a relaxed, unfocused state of mind. It probably seems counterintuitive that one of the best ways to get unstuck is to stop trying for a little while, but it's true. The problem is that it's easy for us to put on mental blinders when we're focusing hard on a problem. I mean, that's pretty much what "focus" means. But by focusing hard, we're exploring only a small portion of the solution space. By unfocusing, our unconscious mind is able to explore and make connections across vast swaths of our experience. To me it's like when you're trying to find a destination on a paper map (remember those?). You can unconsciously become convinced that the city you're trying to reach should be right here! in the upper-left qudrant of the map, so you look at it over and over without success. Then you put the map down and take a deep breath and stare at nothing for a minute, and when you look at the map again the actual location jumps out at you immediately. We've all had the experience of having a sudden insight in the shower; now you have a slightly better understanding of why that happens, and you can employ the technique on purpose. Personally, I will actually take a shower if I'm stuck on something, and it's remarkable how well the technique works. And how clean I am. If you'd like to learn more about the focused and diffuse modes of thinking, check out A Mind for Numbers: How to Excel at Math and Science (Even If You FLunked Algebra). Don't Waste Time Being Frustrated Related to the last section: don't waste time being frustrated with code. Frustration leads us into doing stupid things like re-compiling a program or refreshing the browser with the hope that this time it will be magically different. Use frustration as a signal that there's a gap in your knowledge. Once you realize you're frustrated, it can help to take a step back and clearly identify the problem. If you've written some code that's not working, explicitly explain to yourself or someone else the result that you expected. Use the scientific method and develop a hypothesis for what's causing the unexpected behavior. Then test your hypothesis. Try again, and if a solution still eludes you, put the problem aside and come back to it later. I can't tell you how many times I've thrown my laptop in disgust over a seemingly unsolvable problem, only to look at it the next day and have an obvious solution pop into my head immediately. This happened last week, even. Identify Which Programming Language Aspect You're Dealing With Personally, I find it useful to keep in mind that when you're learning a programming language, you're actually learning four things: How to write code: syntax, semantics, and resource management The language's paradigm: object-oriented, functional, logic, etc. The artifact ecosystem: how to build and run executables and how to use libraries Tooling: editors, compilers, debuggers, linters It's easy to get these four facets mixed up, with the unfortunate result that when you run into a problem you end up looking in completely the wrong place. Someone who's completely new to programming, for example, might start out by trying to build iOS apps. They might try to get their app running on a friend's phone, only to see some message about needing a developer certificate or whatever. This is part of the artifact ecosystem, but an inexperienced person might see this as a problem with how to write code. They might look at every line they wrote to figure out the problem, when the problem isn't with their code at all. I find it easier to learn a language if I tackle each of these aspects systematically, and in another blog post I'll present a general list of questions that need answering that should help you in learning any language. Identify the Purpose, External Model, and Internal Model Whenever you’re learning to use a new tool, its useful to identify its purpose, external model and internal model. When you understand a tool's purpose, your brain gets loaded with helpful contextual details that make it easier for you to assimilate new knowledge. It's like working on a puzzle: when you're able to look at a picture of the completed puzzle, it's a lot easier to fit the pieces together. This applies to languages themselves, and language libraries. A tool's external model is the interface it presents and the way it wants you to think about problem solving. Clojure’s external model is a Lisp that wants you to think about programming as mostly data-centric, immutable transformations. Ansible wants you to think of server provisioning in terms of defining the end state, rather than defining the steps you should take to get to that state. A tool's internal model is how it transforms the inputs to its interface into some lower-level abstraction. Clojure transforms Lisp into JVM bytecode. Ansible transforms task definitions into shell commands. In an ideal world, you wouldn’t have to understand the internal model, but in reality it’s almost always helpful to understand a tool's internal model because it gives you a unified perspective on what might seem like confusing or contradictory parts. When the double-helix model of DNA was discovered, for example, it helped scientists make sense of higher-level phenonema. My point, of course, is that this blog post is one of the greatest scientific achievements of all time. Tutorials often mix up a tool's external model and internal model in a way that’s confusing for learners; it's helpful to be aware of this so that you can easily identify when it's causing you frustration. Spaced Repetition Helps You Remember Spaced Repetition been proven to be one of the best ways to encode new information in long-term memory. The idea is to quiz yourself at ever-increasing time intervals to minimize memory decay using the fewest number of repetitions. The Guardian wrote a great introductory article. Sleep and Exercise Take care of your body! It's more than just a vehicle for your brain. If you want to be able to stay focused and learn efficiently, getting adequate sleep and exercise beats the pants off caffeine and energy drinks. More tips? If you have any useful tips, please leave them in the comments! If you'd like more excellent resources on learning to program, check out the Community Picks: Learn Programming, a community-curated collection of the best books for learning programming. It includes a wide array of subjects, including introductory programming books, books on craftsmanship, and books on soft skills and interviews.

Kubernetes with ELK Setup

about 4 years ago | Rocky Jaiswal: Still Learning

As it turns out this is my 100th post. The first one was published in March 2011 and a lot has changed since then. A lot of new technologies have come in and a lot of my posts have become obsolete. In all these years nothing has created more i ...

Tips and Tricks for Beautifully Simple Mobile App CI

about 4 years ago | Dave Kerr: dwmkerr.com

In this article I'm going to demonstrate some simple tips and tricks which will help you build and maintain beautifully simple mobile build pipelines. These techniques can be applied to different mobile app technologies and integrated into almost any build system: Sample App Index Each tip is demonstrated in the sample apps in

Testing HapiJS with Jest

about 4 years ago | Rocky Jaiswal: Still Learning

HapiJS and React are pretty much my go-to technology choices for web application development right now. While Jest works really well with React, on the HapiJS side I was ...