Updates
- 8/25/2020 – Phase 1 (moving the production site to a dedicated physical server) is now complete
- 9/28/2020 – Phase 2 is now complete
- 12/31/2020 – Phase 3 is now complete
- Currently in Phase 4 (development) – on schedule
Intro
In this post, we share Backpacking Light’s plans for its next phase of website development. Over the past several months, we have been vetting enterprise website development agencies. We have made our final decision and are excited to announce that we are building a new site and plan to launch it in 2021.
First, let’s talk about what’s around the corner, and then what’s on the longer-term radar.
New Site Search Engine
In 2018, we solicited feedback from our membership about site search, and the difficulty members had in finding content in our article library and forums. This remains our users’ #1 pain point.
In 2019, we designed the specs for our new site search and started building the new search engine in the fall. Earlier this spring, we distributed a beta version to Unlimited Members who gave us invaluable feedback about it. We’ve incorporated nearly every suggestion submitted by several hundred Unlimited Members, and we’re now getting ready to roll it out to everyone! We will be replacing the old site search with this new search engine before the end of the month.
Here’s a preview of what’s coming:
What’s Next?
We invested in a custom-built platform and server architecture in 2007 and ran on that platform until 2015. In mid-2015, our hosting and development company abruptly announced that they were ceasing operations, and we only had a few months to build a new site. That site was developed rapidly and launched in November of 2015. The primary goal of the 2015 transition was to maintain continuity of our website and content – it was not to be a long-term solution. But at the time, we didn’t have the resources to manage a new ground-up development effort, so we have had to make do with that site until now.
The primary challenges with that site were related to security and speed. It became clear that we would need to invest substantial resources to lay the foundation of a highly secure, fast, and scalable server architecture. So 2016-2017 became a period of saving money to pay for the transition to our first enterprise server environment. The focus of our web development efforts in 2018-2019 has been primarily on developing and optimizing that server architecture, so we could begin website redevelopment/redesign efforts in 2020.
BPL has close to two million forum posts, articles, member profiles, and other content records stored in 175+ different databases. These databases are interdependent and accessed by several hundred users online at a time with complex queries. Maintaining stability and speed is an enormously challenging problem that’s a balancing act between server resources, database optimization via regular maintenance, and continued new development to maintain state-of-the-art data access and delivery code. As our current site ages, it becomes more difficult to maintain because we didn’t have the time or resources to develop scalable architecture in 2015.
And so, here’s what’s next.
1. We have increased our budget for website support and maintenance and have hired a new website support agency.
Maintaining the website effectively, including fixing bugs that pop up when code updates are installed, is a complicated and expensive process due to the complexity of the website and the extent of its customization. On an enterprise site like ours, bug fixes require validation and source analysis, a fix proposal, code writing, code review, testing on a staging server, migration to the production server, and testing on the production server. We have to maintain very strict standards for change control to maintain the security, speed, and stability of the site.
We retained a new website support services agency that will be providing site monitoring, security, and maintenance services for us. Because of the complexity and size of the BPL website, there is a substantial onboarding/training/transition period, and they won’t be up to full speed until later this summer. This is the agency that will be handling our site maintenance including routine code updates and bug fixes. Moving to an outside support agency comes with a substantial cost (consider that at least 10-15 hours per month of developer hours are required to maintain BPL at rates of $150 to $200 an hour and you get an idea of what sort of commitment this is).
2. We are adding an additional community moderator who will focus on website issues.
We are seeking an additional moderator from our community who will help us manage bug reports and communications between the BPL community and the website support and development team. This moderator’s responsibilities would include:
- Monitoring and moderating the Website Issues Forum.
- Acknowledging new bug reports, logging them in BPLTrac, and communicating them to the website support and development staff (here’s how we process bug reports).
- Two-way communications with our staff in our company’s internal Slack channel, so the moderator can participate in real-time internal communications about website issues.
- Communicating progress back to the community with updates about bug reports and other website issues.
If you’re interested in this role, have a reasonable understanding of how web development and web technologies work, have a diplomatic disposition, and are able to communicate technical information to a lay audience, please drop me a note at publisher@backpackinglight.com.
3. We are about to finish our 2-year transition to enterprise server architecture.
Site security, speed, and stability comprise the most important foundational pillars of a healthy site, and we knew we had to get these things right before embarking on any major development projects. Since 2015, we’ve experienced issues with speed and stability. And in the past 18 months, attempts at breaching our site security have grown more than tenfold. We now block 30% of attempted traffic to the BPL site. This traffic is the direct result of malicious bots and known hacking networks attempting to steal passwords and other user data, overwhelm server resources, and inject malware into the site. Such is the nature of the new internet. It’s normal, but serious and requires money and infrastructure to combat it and protect our user data. Security will always be our #1 priority.
Last month, we took another step in what has been a 2-year transition period to new server architecture. We now host BPL on a custom-configured enterprise-class server with enterprise-grade security systems in place to protect user data and site functionality. This was a critical step to achieve the security, speed, and stability required for scale, and to proceed with further development.
We have one more phase in building out our enterprise server architecture:
Decoupling database operations from the rest of the site (where database ops are processed on a different server than the rest of our ops)In early August after tests on a dedicated vs. decoupled server, we made the decision instead to upgrade our server architecture to a dedicated server with a local database. While more expensive, it will result in substantially better performance. This upgrade is schedule for August 7-10, 2020- Finishing installation of state-of-the-art security protocols to take advantage of the security policy compliance opportunities afforded by the most modern browsers.
This is the focus of 3Q2020.
Once this foundation of security, stability, and speed is established, we can then proceed to the next era of backpackinglight.com.
4. We have started building the next Backpackinglight.com website.
The site we launched in 2015 was an attempt to re-create the functionality of the pre-2015 site with minimal UI (user interface) disruption to our community. We knew we couldn’t create a modern user experience with the resources and time constraints that we had. As such, the BPL site as it exists today is inelegant, with an aging user interface. In particular, forums have limited functionality, ad injection clutters the site in unpredictable ways, the article reading experience is clunky, managing member accounts and profiles is difficult, online course delivery is antiquated, and styling is inconsistent across site sections.
Our mission for the next site is:
Completely update the user interface, site design, and functionality so the user experience is enjoyable – refreshing, easy, fast, intuitive, immersive with our content, and able to connect members of our community in more meaningful ways.
We have retained the services of a website development agency that specializes in custom-built sites for media publishers with large memberships and forum communities, online courses, and robust content publishing/content management platforms. We’ve reviewed their work, worked with them on preliminary scoping, and conducted due diligence with several of their previous clients. They are among the very best UI/UX designers of membership communities in the world.
Our plan is to launch the new site in May 2021.
Here’s our development timeline. We will update our membership as we progress, and let you know which items have been completed so you can see exactly where we’re at on our schedule.
Combined with our existing enterprise server architecture, the next website will usher in a new era for BPL. It will give us the opportunity to:
- Implement modern website design and user interface standards to vastly improve the user experience at the site – allowing us to maintain modernity into the future without major redesigns of underlying architecture;
- Create a platform with scalable maintenance and speed in response to continued growth;
- Offer features and functionality that will enhance member experiences at the site, including community, content discovery/curation, site search, and online course delivery.
It became clear in 2015 that we needed to put a plan in place to create a state-of-the-art website that served the needs of our community. We’re excited to enter this next phase of our transition because this is the phase that results in dramatic improvements in user experience for our community.
The reasons for the lengthy time frame to get to this point are simple:
- This final piece of development – the new site design – carries a substantial price tag. It will require 300 to 400 hours of high-level engineering by outside developers at rates of $120 to $180 an hour. That’s on top of annual server and tech maintenance costs that are also very expensive. I’m debt-averse and am committed to paying cash for a new website. That means money has to be saved. It takes time to save that much money, and time to ensure that we have enough operating cash flow reserves to properly commit to ongoing maintenance when we take on these new recurring expenses.
- It takes time to research, study options, and make decisions that have the potential to impact our operations for several years into the future.
It’s easy to burn money on development efforts that have short-term impacts, but doing so can result in dramatic losses of long-term opportunities that require substantial cash reserves. This is the universal challenge of all entrepreneurial endeavors. It didn’t take long before we realized that the 2015 site was not going to be a long-term solution, so I prioritized saving cash for tomorrow’s build rather than trying to continue hacking together a disjointed site and missing out on this current opportunity. The downside of this unpopular business management strategy, however, is that it frustrates our users when we make the decision not to make their requests for site features and functionality a higher priority during periods where we are saving cash for large projects. I understand that frustration.
Your Feedback is Essential: Take the Survey
We will be soliciting feedback from our community throughout this process.
Today, we’d like to invite you to participate in this very short survey so we can gather feedback that will help us during the design phase. This information goes straight to the development agency, so it’s critical that if you want your feedback to be heard and processed by the folks that are actually building the new site, that you do so in this survey!
Thank you for your ongoing support. We are looking forward to bringing you the next version of the BPL website!
Discussion
Become a member to post in the forums.
In this post, we share about Backpacking Light’s plans for its next phase of website development. Over the past several months, we have been vetting enterprise website development agencies. We have made our final decision and are excited to announce that we are building a new site and plan to launch it in 2021.
” We now block 30% of attempted traffic to the BPL site. This traffic is the direct result of malicious bots and known hacking networks attempting to steal passwords and other user data, overwhelm server resources, and inject malware into the site.”
I knew the web was a rough place, but I did not know it was this bad!
Hang in there.
Cheers
We get brute force (e.g., bot login attempts) attacks – each one from a different IP – at least once per second. It’s nuts.
And they love to target users who have easy-to-guess passwords or ones that have already been compromised. Play your cards close when it comes to passwords.
The new search looks slick as hell. This whole writeup gives me a lot of confidence in the future of BPL. Thanks for keeping us updated!
Roger
“I knew the web was a rough place, but I did not know it was this bad!”
It’s mostly script-kiddies running fairly innocuous of-the-shelf hacks, plus generic Russian, North Korean and Chinese hack-bots looking for unpatched vulnerabilities.
Nowadays the developers, operating systems, languages, frameworks and hosting environments are far more security conscious – it’s actually easier to write a relatively secure site than it used to be, despite the volume of attacks.
Of course nothing on the open web is truly secure, but for a small operation like this you can pretty much ensure that the cost of a bespoke hack would be greater than the value of the data, so you don’t attract the attention of the real pros.
And we thought it was bad when we had the shoe-bots!
Cheers
PS: yes, I know who is doing the attacks. They annoy me.
With my tiny blog I now have three services to protect my site. Two I pay for and cost me $400 per year. My site has had 3.5 million page views in 12 years, meaning it is a very small blog. Keep in mind my site is free, so I had to finally include ads to pay for security, web hosting fees, etc.
From the beginning I turned off the viewer comments feature. For several years I would sometimes get hundreds of comments per day, but none could be published, but they took up resources and time to delete. I was finally able to block that.
A couple years ago or so, someone was able to add code to a widget, which showed the temperature in Palm Springs, on my site that would mine bitcoin using a visitor’s computer. That was a mess.
Brute Force attempts the past two years caught by just one of my security programs.
And so it goes. Another service blocks IP address that attempt to log in to the admin site. That goes in batches by country. Eastern Europe, China, Indonesia, etc. Each of my three security services catches things the other two don’t.
From all of this I can really appreciate the time, money, and effort Ryan has to constantly invest in BPL.
I do have a sort of bushwalking blog, an old one, but it is hard-coded in HTML and has no 3rd-party SW and no user comments. I made it strictly Read-Only.
Cheers
Roger – Sometimes, old-school is the best school :)
Glad to see BPL’s growth and looking forward to the enhancements. I don’t know if it’s related to those, but in the last few days I’m experiencing link misdirection and lack of permissions – I’ll click on a forum notification, only to be redirected to a different thread or to see a message that I don’t have permission to see those threads (I’m a longtime Lifetime/Unlimited member).
EMOJIS! WE WANT EMOJIS!
(Well maybe not “The Finger” emojis oe poop emojis or kiss emojis but just a nice selection of face emojis – in all skin colors.)
Wait – what? You can’t use emojis? 🤔 we should be able to fix that 🙏
You’re going to put an emoji button menu in the formatting bar?
^ I hope not…
But on the Mac, hold down the Control and Command keys and the spacebar all at one time. This brings up the emoji screen. Select the one you want and voila…🐾
I hope we don’t once again lose a bunch of members due to subscription snafus. Should we expect all our saved links, subscriptions etc to be lost when the change is made. I lost so much it’s still annoying to think about.
Keep your user data, or get an emoji button? Can’t have it all (wink) 😉
Become a member to post in the forums.