Future of the web

I recently had the opportunity to attend WordCamp Asia in Taipei. It was my first time in a WordPress community event. One of the key takeaways for me was that WordPress adoption was in decline. Not only that, behind closed doors, I got to hear that number of websites created on the web overall is slowing down as well (no data was shared unfortunately… but I believed the source)! This got me thinking… does WordPress no longer meet the needs of people? Or perhaps, websites are going away..?

Keynote presentation from Noel Tock: Future of WordPress

Search for answers like “can dogs eat mango” or “do butterflies pee” (yes, I’ve searched for both recently), you can easily see how Google’s negative influence on the web. Job of an indexer is to make sense of content and surface it appropriately. However, search is the window to the web, shifting some power from authors to the indexers of content. The web is, at least partially, shaped in the image of Google.

Despite search being the dominant player on the web, authors still get to style and present content in ways that reflect their branding and style. Websites are a reflection of an author’s intent. They get to take the consumer on a journey. Search is part of how content is organised. However, all of this lies in the hands of the author.

I asked ChatGPT the same questions above to get answers. Answers were instant. No bullshit ads or content hidden amongst a sea of SEO text. I was only regretful that I hadn’t done that as my first step. I’m still building my muscle memory to open ChatGPT instead of Safari.

There is something interesting about searches done through LLMs. LLMs drop all hints of what the original authors of this content intended to convey or how they wanted to present the information. Content is reduced all the way to what I personally wanted to know.

AI is becoming the translation layer between what authors created and what the consumer was looking for. This is causing a fundamental shift to how we think about the web. The presentation responsibility shifts from the author to consumer, reducing the burden on something like a website.

There are lots of down sides to this, of course. Content creators don’t get enough credit (hence, lawsuits) and we don’t read how content was intended to be consumed. We miss out on the motivations and stories behind what fuelled great writing or artwork. Yet, the significant amount of trash on the web today will move more people into this mode of using something like ChatGPT to discover content.

This isn’t a new phenomenon either. Google Maps is a good example of providing a different window to content around the web. That too, has a lot of complaints from content owners. However, Google Maps is something Google designs to be a specific lens into a type of content. It isn’t as personal to me specifically.

Web browsers have been the window to the web. If you had the opportunity to watch Halt and Catch Fire (season 2 and 3), Joe MacMillan has this epiphany. They don’t have to create their own network to get users. They just need to create a window to what is out there in other networks and servers with HTTP requests. It is so powerful. A web browser’s job is to present content an author has prepared using HTML and CSS. JavaScript makes things more fun and interactive.

Entering the AI age, the concept of a “web browser” takes a different shape. My browser should be different to yours because I, as a consumer, get to control how content is presented to me. Browsing will be deeply personal. I have the power to decide what I want to see. My bot / browser / companion / OS / computer / whatever I use… will understand me and shape content to help me easily consume it. It will help me make sense of the world.

It is started to happen: Arc Search

In fact, this will go so much further in the future. Bots will talk to other bots to learn about things and bring final outcomes back to the consumer. We’ll build more and more purpose driven applications/models that reduce vast amounts of information to simple consumable chunks.

It will be an absolute mess before we get it right. But it will be the begining of something new.

We may very well end up making a lot less websites in the future. This is not cause for alarm. Part of my current job includes looking at products like cPanel hosting. People aren’t reaching for those tools as much over time. However, more content is being hosted on the Internet than ever before, just not in traditional hosting. Social media collects incredible amount of content from users. These changes on the web will make consuming more content possible and CMSes and content creators will still be super important. They just might not exist in the current shape and form.

What will be important here is to figure out how do we continue to have a web that isn’t a series of walled gardens like Facebook, TikTok, Reddit, X and so on. Internet should be open. It should potentially be free to a large extent too. The benefits of it being that way are obvious: the free Internet has progressed our society at an incredible pace.

There’s a lot to figure out in this future. Its success will be determined by whether we can solve these problems.

  • How would we give credit and incentivise creators to continue to make great things? Imagine technology like ActivityPub and Activity Streams will play a key role in the distribution of content and credit.
  • How would we make the economics work? Advertising powers much of the Internet today. Websites can mix ads with content to make revenue. Reddit, X and others have closed their APIs and websites to crawlers that look to create LLMs to protect revenue. Would we end up with walled gardens of content that takes a fee to browse? Wouldn’t be a very open Internet anymore.
  • How do we evaluate such an unpredictable medium to be accurate enough or safe to use? Explainability will become important.

As someone who is obsessed with user experience, this is really exciting to think about. This fundamental shift will bring new ways to explore and consume information. There is so much innovation opportunities ahead of us.

Euclidean distance vs Cosine Similarity for text searches

Cosine similarity is the angle between two points, while euclidean distance is the actual distance between two points. Cosine doesn’t care as much about magnitude so does well for different lengths of data (think how far into a specific direction something is as long as they are in that direction).

Thinking beyond your team

Recently, couple of my teams decided to combine their codebases. It was essentially the same application, written twice over to do the UI for a wizard that went down 2 paths. Their backends do very different things. However, the customer sees a very similar interface.

After having done the migration, there have been some gripes. There is a lot of waiting between time zones. Ownership isn’t clear. Operational setup hasn’t been worked out. Perhaps the biggest issue was that there are two teams for a single codebase. It seems to trip people up when it comes to figuring things out.

So I’ve been asking this one question over and over:

Imagine the other team reported to you (or you were part of the same team). How would you solve this?

Another version of this question:

Imagine you had 15 people in your team across 2 locations. How would you run your scrum processes?

Once you remove the artificial manager/team/pod based barrier, people seem to work it out. This little mental barrier is the root of a lot of problems when collaborating. It is really important for people to identify their tribe. For most people, their tribe is their scrum team. Increasingly, that isn’t good enough to get the best results. At least in my team, we need to expand our view of what our tribe is.

Do all software engineers care about customer impact?

I got into a fun chat with my friends about unengaged people at work. Do software engineers care about salary, solving a problem technically (discovery), shipping, or does that work has to have impact in someone else’s life? Of course, different people find joy in many points within this spectrum of outcomes.

Interesting question surfaced: is it that some people don’t find joy in having impact in others or have they just not had the opportunity to make that deep connection to their work? I mean, nothing wrong with just finding joy in solving problems, right?

I referenced 2 stories from my past.

I used to go to church multiple times a week and be involved in many religious activities. I was happy and I never really thought past the bubble I was in. Then my circumstances changed with where I lived, my closest friends’ life circumstances etc. It led to me inspecting the bigger “why’s” of life. As a result, I incrementally focused my energies elsewhere and dropped out of church.

Second story was just me as a software engineer. I loved experimenting with new technologies and making fun little UI apps. As a teenager, I had an obsession with how traffic lights worked. I found it really fun to code road junction in Java Swing UI. It was a fun way to learn Java but also… just liked figuring out how things worked.

Then I made a jump to building some iOS apps with my brother. I’ve written about this before. Watching us have direct impact in someone else’s day was just incredible. I was working for Vigil Systems at the time where we guaranteed 15% reduction in public transport accidents (sketchy memory on this metric). These things mattered to people. The direct interaction and knowing how my work helped people was a wonderful feeling.

After that, I always wanted something more. It wasn’t just enough to write good code or ship things quickly. I wanted to know my code made a difference and it helped people to make time or be better off in some way.

So, is it that some don’t get to experience this moment in time that helps them break out and think beyond their Jira cards? If you are starting off in a big technology company, it is really difficult to have this direct connection to customers to begin with. Most developers I work wth never had to explain to a customer how to find a feature in the interface they coded in. By the way, it is extremely frustrating to see people struggle with what you’ve built.

This image came to mind… this is Maslow’s hierarchy of needs:

Are we just thinking about people in different stages of this pyramid? Does everyone have the opportunity to attain self-actualisation? Does self-actualisation mean having direct impact in others?

I think it is pretty important to generate that connection at some point. But we should remember that sometimes, we find meaning in having impact in others’ lives… and sometimes, discovering is just as important. The very act of making and having fun can lead to things. We don’t have a clear purpose in life. So these competing forces — this chaos — keeps us vibrant and moving forward.

Whatever it is for you, I am hoping you find engagement and joy at work. It helped you stay motivated. If your current work doesn’t give you that, find something that does.

Related article to read: How to burnout a software engineer, in 3 easy steps.

Barry to Google on "increasingly aggressive Google"

Worth the read (images in the post). Having been at Expedia, this issue is all too familiar. I hope someone does something.

Google should profit share for the data they use to run their business, much like OpenAI should for its data for LLMs.

Twitter link

Play the long game with people, the short game with outcomes

Recently, @henry sent over my 2020 performance evaluation I had shared with him. I was just about to go from Principal to Director role. I'm still trying to get good at this. Such wonderful advice from @AC. Thanks boss!

Play the long game with people, the short game with outcomes. Knowing how much to push, when to push and how hard is an art. With technology decisions its pretty easy, there is often a right answer and a wrong answer, often the wrong answer is no decision. However with people its a bit different and Dineth can show his frustrations at times when someone is not keeping up. Dineth is great at setting a high bar, this year he will have the opportunity to set a high bar, coach and help his team to hit it but also Dineth will need to make hard calls when someone isn't going to be able to make it. This takes time and practice and I would like to see this be a big focus of his this year.

Car crashes and sticking to your lane

My team recently made the decision to not have QA role as part of experience engineering. There were a lot of complains on increased workload and not wanting to do testing, including some valid reasoning around testing not being an area of expertise.

I’ve heard similar complaints when we spoke about creating dashboards and alerts for observability — “backend crap”. When I was a manager a while back, I even had someone in my team tell me he can’t work without a TPM writing the requirements down in the card. Requirements gathering is apparently not part of the job.

I recognise that there is discomfort in breaking out of what we are good at. Being a coder, there is a level of certainty to what we do. To think about customer behaviours and ways to break a system is adding uncertainty. There is a lot of resistance to do what needs to be done. Everyone likes to stick to their lanes.

Unfortunately, hard boundaries make for shit software products. People learn to stop asking why things are done/made a certain way. A level of disengagement sets in. Boundaries take people away from what matters the most: having [customer] outcomes.

I was trying to explain this recently with the metaphor of a car crash. Maybe this will resonate.

Imagine you are in the passenger seat of a car. Your friend is driving you back from a day at the beach. Beating sun and the long drive has made him tired. You aren’t much of a confident driver, so you can’t really offer to help. Tired eyes give way and your friend falls asleep at the wheel. You try to wake them up but Morpheus has trapped him in the land of dream.

What do you do?

Do you say, “not my area of expertise” and watch yourself go hurling into a sidewall? It would be pretty sensible to grab the steering wheel and find a way to get both of you to safety.

This is what most work is like. We constantly have to go out of our comfort zones to make outcomes happen. That is the entire value of an engineer… they solve the necessary problems to make outcomes happen.

2 things to leave you with:

  1. When you hire for a team, mix experts with generalists.
  2. Explain to everyone that in your team, they’ll have to step out of their comfort zone to make things happen. That they are expected to solve problems no matter where it takes them.

Anticipatory Failure Determination

[AFD] has the objective of identifying and mitigating failures. Rather than asking developers to look for a cause of a failure mode, it reverses the problem by asking developers to view the failure of interest as the intended consequence and try to devise ways to assure that the failure always happens reliably

Interesting way to discover unknowns and side step things like denial.

https://www.npd-solutions.com/afd.html

Thanks Connor for the share...