Among the advantages of a highly distributed system with endless edge points are that you can outsource data collection to a universe of locations, and even include them in your workflow, thereby expanding your operations. The challenges are when you have endpoints that contribute to your organization and systems, but you don’t exactly trust. They can be newcomers that you don’t know enough about, or entities with a history of misusing the data inclusion to your systems give them access to. You want the value they create, the information they amass and gather to be copied from the edge up the levels of your system, but you don’t want to give too much for that value or pay for it in the form of greater risk. Filtered replication is the art of enabling nontrusted edge points to access your system in a limited manner, replicating the information they produce in a nontrusted format.

Yesterday I posted about Parler banning and the likely impact of that, both legally and in terms of the technical details. My expectations is that new actors will step in to fill the existing demand created by the current social network account suspensions. I had spent some time thinking about the likely effects of this, and I think that it will lead to some interesting results.

A new social network will very likely rise as a result of those actions. That network would have to be resilient for de-platforming issues. That means that it cannot assume that it can run on any of the cloud services, at least not as normally understood by today’s standards. That means that we are likely to see one of two options:

• Fully distributed systems – independent nodes collaborating with one another to create a network. Each node may be host and operated independently. Similar to how torrents work and other fully distributed P2P systems.
• Distributed infrastructure – a set of servers that are running on behalf of a single entity, but are spread over multiple vendors and locations. The idea is that the shutdown of a single or multiple vendors will have little impact, because of distribution of effort.

The first option is probably something like Mastodon, but I would really like to see a return to blogs & RSS as the preferred social network. That has the advantage of a true distributed model without a single controlling actor. It is also much lower cost in terms of technology and complexity. Discovery of new blogs can be handled via recommendations, search, etc.

The reason I prefer this option is that I like to blog . More seriously, owning your own content and distribution platform has just become quite important. A blog is about as simple a piece of software as you can imagine. Consuming blogs is an act that require no publication of personal information, no single actor that can observe everything you do, etc.

I don’t know if this will be the direction, although it is my favorite one. It is possible that we’ll end up with Mastodon empire, with many actors creating networks of servers which may or may not be interconnected. I can see a future where you’ll have a network of dog owners vs. cat owners, but the two aren’t federated and there are isolated discussions between them.

Given that you could create links from one to the other, I don’t think we have to deal with total echo chambers. Consider a post in the cats social network: The dog owners are talking about the chore of having to go for walks at “dogs://social.media/walks-are-great”, that is so high maintenance, the silly buggers. 

That would create separate communities, with their own rules and moderation. Consider this something like subreddits, but without the single organization that can enforce global rules.

The other alternative is that a social network would rise with a truly distributed backend that is resilient to de-platforming issues. From an outside perspective, this will present as something to the existing social networks. That has the advantage of requiring the least from users, but it is a non trivial technical challenge.

I prefer the first option, but I believe it is more likely we’ll end up with the second. The reason for that is monetization strategies. If you have a many different actors cooperating to create a network, there is a question on how you pay for that. The typical revenue model for social network is advertising. That doesn’t work so well where there isn’t a single actor that can sell ads (and track users).

That said, it would be much faster and easier to get started with the first option and it may be that we’ll end up there with the force of inertia.

I’m writing this post at a time when Donald Trump’s social media accounts were closed by Twitter, Facebook and across pretty much all popular social networks. Parler, an alternative social network has been kicked off the Apple and Google app stores and its AWS account was closed. It also appears that many vendors are dropping it and it will take significant time to get back online, if it is able to do so.

I’m not interested in talking about the reasons for this, mind. This is a hot topic political issue, in a country I’m not a citizen of, and I have no interest in getting bogged down with the political details.

I wish I could say that I had no dog in this fight, but I suspect that the current events will have a long term impact on the digital world. Note that all of those actions are taken by private companies, on their own volition. In other words, it isn’t a government or the courts demanding this behavior, but the companies’ own decision making process.

The thing that I can’t help thinking is that the current behavior by those companies is direct, blatant and very much short sighted. To start with, all of those companies are working on global scale, and they have just proven that they are powerful enough to rein in the President of the Unites States. Sure, he is at a lame duck status currently, but that is still something that upset the balance of power.

The problem with that is that while it would appear that the incoming US administration is favorable to this course of action, there are other countries and governments that are looking at this with concern. Poland is on track to pass a law prohibiting the removal of posts in social media that do not break local laws. Israel’s parliament is also considering a similar proposal.

In both cases, mind, these proposed laws got traction last year, before the current escalation in this behavior. I feel that more governments will now consider such laws in the near future, given the threat level that this represent to them. A politician is this day and age that doesn’t use social media to its fullest extent is going to be severely hampered. Both the Obama and the Trump campaigns were lauded for their innovative use of social media, for example.

There are also other considerations to ponder. One of the most costly portions of running a social network is the monitoring and filtering of posts. You have to take into account that people will post bile, illegal and obscene stuff. That’s expensive, and one of the reasons for vendors dropping of Parler was their moderation policies. That means that there is a big and expensive barrier in place for future social networks that try to grow.

I’m not sure how this is going to play out in the short term, to be honest. But in the long term, I think that there is going to be a big push, both legally and from a technical perspective to fill those holes. From a legal perspective, I would expect that many lawyers will make a lot of money on the fallout from the current events, just with regards to the banning of Parler.  I expect that there are going to be a whole lot of new precedents, both in the USA and globally.

From a technical perspective, the technology to run a distributed social network exists. Leave aside currently esoteric choices such as social network on blockchain (there appears to be a lot of them, search for that, wow!), people can fall back to good old Blog & RSS to get quite a bit of traction. It wouldn’t take much to something that looks very similar to current social networks.

Consider RSS Bandit or Google Reader vs. Twitter or Facebook. There isn’t much that you’ll need to do to go from aggregation of RSS feeds to a proper social network. One advantage of such a platform, by the way, is that it allows (and encourage) thought processes that are longer than 140 characters. I dearly miss the web of the 2000s, by the way.

Longer term, however, I would expect a rise of distributed social networks that are composed of independent but cooperating nodes (yes, I’m aware of Mastodon, and I’m familiar with Gab breaking out of that). I don’t know if this will be based on existing software or if we’ll end up with new networks, but I think that the die has been cast in this regard.

That means that the next social network will have to operate under assumed hostile environment. That means running on multiple vendors, taking no single point of failure, etc.

The biggest issue with getting a social network off the ground is… well, network effects. You need enough people in the network before you start getting more bang for the buck. But right now, there is a huge incentive for such a network, given the migration of many users from the established networks.

Parler’s app has seen hundreds of thousands of downloads a day in the past week, before it was taken down from the app stores. Gab is reporting 10,000+ new users an hour and more users in the past two days than they had seen in the past two years.

There is a hole there that will be filled, I think. Who will be the winner of all those users, I don’t know, but I think that this will have a fundamental impact on the digital world.

My talk to the Azure Israel about Cosmos DB is up (in Hebrew)…

I’ll be speaking at the Azure Israel group at the end of this month, you can register here:

In this session, Oren Eini will discuss how you can use Azure Cosmos DB in your application. We'll go over data models, proper design, and best practices for building applications using Cosmos DB.
We are going to look into some of the common pitfalls with regards to both performance and cost.

Yesterday I had a great chat with Rodrigo, it went on for almost two hours and I got to show off some cool features in RavenDB as well as to talk a little about my history and motivation.

As usual, I would love your feedback.

I run into this link, which is Lambda school offering to place a student of theirs at your company for up to four weeks without having to pay them or the student. They market it as: Add talented software engineers to your team for a 4 week trial at no cost for your company if you do not hire the fellow.

Just to point out, this is not relevant to me or mine, so I don’t have a dog in this fight, but I run into this offer because of this:

“This program is part of Lambda School for the Fellow and as such, is not paid.” https://t.co/Dko0zgy05E

— Scott Hanselman (@shanselman) November 19, 2020

Scott then wrote: “Pay people for their work. Pay interns.”

I think that this ties closely together with the previous posts on money and open source. On the one hand, I think that this is a pretty good offer for a student, because they are able to get over what is likely to be their biggest hurdle, actual experience in the field. It also lowers the bar on actually getting their foot in the door, after which the second step is likely to be much easier.

On the other hand, for the company, it is a great way to do test runs for entry level positions. Hiring developers is hard, and being able to trial run for a few weeks is much better than trying to predict the candidate’s ability based on a short interview.

On the gripping hand, however, there are a bunch of issues here that are rife for abuse. This also means that the only people who’ll get to this program are those who are able to actually go a month or so without pay. Or that they will need to do this and their regular work as well.

An “easy” way to fix this would be to pay at least minimum wage to the students, of course. The problem is that this would curb a lot of the advantages of the program. I’ll refer you to Dan Ariely and the FREE! experiments for the reasons why. From the company’s perspective: Paying a minimum wage or getting an employee for free is pretty much the same thing. But there is a lot less process to getting someone for free. And once there is a foot in the door, it is a lot easier to convert internally.

What I wish was possible was to be able to hire people (at or near market rate) for a short amount of time and then decide if you want to keep them on afterward. The idea is that actually seeing their work is a much better indication of their capabilities than the interview process. That reduce the pressure to perform during an interview, gives candidate far better chance to impress and show off, allow them to learn, etc.

This is technically possible but not actually feasible in almost all situations. Leaving aside labor laws, consider the employee’s perspective in this case. If they are already working, going to another company and doing a “trial run” which can be unsuccessful is a very powerful reason to not go there. A company with the kind of reputation of “they hired me for a month and then fire me” is going to have hard time getting more employees. In fact, being fired without a few weeks or months of getting hired is such a negative mark on the CV that most people would leave it all together. Because of this, any company that want to do such trail runs cannot actually do so. They have to make the effort to do all the filtering before actually hiring an employee and reserve firing someone after a short time for mostly egregious issues.

The internship model neatly works around this issue, because you have a very clear boundaries. Making it an unpaid internship is a way to attract more interest from companies and reduce the barriers. For the student, even if they aren’t hired at that place, it gives actual industry experience, which is usually a lot more valuable in the job market.  Note that you can pay the grocery bill with Reputation bucks, it just takes a little longer to cash them out, usually.

The unpaid internship here is the problem, for a bunch of reasons. The chief among them is that making this free for the companies open this up for abuse. You can put controls and safeguards in place, but the easiest way to handle that would be to make it so they pay at least minimum wage to avoid that. The moment that this is paid, a lot of the abuse potential go away. I can imagine that this would be a major hassle for the school (I assume that the companies would rather pay an invoice rather than hire someone on for a short while), but it is something that you can easily do with economies of scale.

The chief issue then, however, would be that this is no longer free, so likely to subject the students to a much harsher scrutiny, which defeats the purpose of getting them out there in the field and gaining real experience.  That is also a problem for the school, I think, since they would have to try to place the student and face a bigger barrier.

Turning this around, however, consider that this was an offer made not for a company, but for open source projects? A lot of open source projects have a ton of work that needs to be done which gets deferred. This is the sort of “weeding the garden” that is usually within the capabilities of someone just starting out. The open source project will provide mentorship and guidance in return for this work. In terms of work experience, this is likely to be roughly on the same level, but without the option to being hired at the end of the four weeks. It also has the advantage of all the work being out there in the open, which allows potential future employers to inspect this.

Note that this is roughly the same thing as the offer being made, but instead of a company doing this, there is an open source project. How would that change your evaluation? The other aspects are all the same. This is still something that is only available for those who can afford to take a month without pay and still make it. From the student’s perspective, there is no major difference, except that there is far less likelihood for actually getting hired in the end.

I run into this tweet:

Remember when open source was about "I'll build this cool feature using this OSS project as the base and contribute it upstream" and not "my multibillion corporation needs this feature so I'll demand it for free from the maintainers"

— Igal Tabachnik (@hmemcpy) November 7, 2020

I wanted to respond to that, because it ties very closely to the previous post. As I already said, getting paid for open source is a problem. You either try to do that professionally (full time) or you don’t. Trying to get hobbyist amount of money from open source is not really working. And when you are doing this professionally, there is a very different manner of operating. For this post, I want to talk about the other side, the people who want to pay for certain things, but can’t.

Let’s say that Jane works for a multibillion dollar company. She is using project X for a certain use case and would like to extend its capabilities to handle and additional scenario. We’ll further say that this project has a robust team or community behind it, so there is someone to talk to.

If the feature in question isn’t trivial, it is going to require a substantial amount of work. Jane doesn’t want to just bug the maintainers for this, but how can she pay for that? The first problem that you run into is who to pay. There isn’t usually an organization behind the project. Just figuring out who to pay can be a challenge. The next question is whatever that person can even accept payments. In Israel, for example, if you aren’t an independent employee, there is a lot of bureaucracy you have to go through if you want to accept money outside of your employer.

Let’s say that the cost of the feature is set around 2,500$ – 7,500$. That amount usually means that Jane can’t just hand it over and claim it in her expenses. She needs someone from Accounts Payable to handle that, which means that it needs to go through the approval process, there should be a contract (so legal is also involved), they might be a required bidding process, etc.

The open source maintainer on the other side is going to get an 8 pages contract written is dense legalese and have to get a lawyer to go over that. So you need to cover that expense as well. There are delivery clauses in the contract, penalties for late delivery, etc. You need to consider whatever this is work for hire or not (matters for copy right law), whatever the license on the project is suitable for the end result, etc. For many people, that level of hassle for a rare occurrence of non life changing amount of money is too much.  This is especially true if they are already employed and need to do that on top of their usual work.

For Jane, who would like her employer to pay for a feature, this is too much of a hassle to go through all the steps and paperwork involved. Note that we aren’t talking about a quick email, we are probably talking weeks of having to navigate through the hierarchy, getting approval from multiple parties (and remember that there is also the maintainer on the other side as well).

In many cases, the total cost that is involved here can very quickly reach ridiculous levels. There is a reason why in many cases it is easier for such companies to simply hire the maintainers directly. It simplify a lot of work for all sides, but it does means that the project is no longer independent.

I run into a (private) tweet that said the following:

Is there a way to pay for a feature in an opensource project in a crowdfunded manner with potential over time payouts? I would love to pay someone to implement a feature I really want, but I won't be able to pay it all.

I think that this is a very interesting sentiment, because the usual answer for that range between no and NO. Technically, yes, there are ways to handle that. For example, Patreon or similar services. I checked a few of those and found LineageOS – 205 Patrons with 582$ monthly.

There is also Librapay, which seems to be exactly what the tweet is talking about, but…  the highest paid individual in there is paid about under a thousand dollars a month. The highest paid organization is bringing in about 1,125$ / month.

There are other places, but they present roughly the same picture. In short, there doesn’t seem to be any money in this style of operation. Let me make it clear what I mean by no money. Going to Fiverr and sorting by the cheapest rate, you can find a developer for 5 – 10$ / hour. No idea about the quality / ability to deliver, but that is the bottom line. Using those numbers (which are below minimum wage) gives you not a lot of time at all.

A monthly recurring income of 500$ – 1,250$, assuming minimum wage, will get you about a week or two of work per month. But note that this is assuming that you desire minimum wage. I’m unaware of anywhere that a developer is charging that amount, typical salaries for developers are in the upper tier. So in term of financial incentives, there isn’t anything here.

Note that the moment you take any amount of money, you lose the ability to just mute people. If you are working on open source project and someone come with a request, either it is interesting, so it might be picked up, or it isn’t. But if there is money involved (and it doesn’t have to be a significant amount), there are different expectations.

There is also a non trivial amount of hassle in getting paid. I’m not talking about actually collecting the money, I’m talking about things like taxes, making sure that all your reports align, etc. If you are a salaried employee, in many cases, this is so trivial you never need to think about it. That on its own can be a big hurdle, especially because there isn’t much money in it.

Counter point to my entire post is that there are projects that have done this. The obvious one is the Linux kernel project, but you’ll note that such projects are extremely rare. And usually have had a major amount of traction before they managed to sort out funding. In other words, it got to the point where people were already employed full time to handle such projects.

Another option is Kickstarter. This isn’t so much for recurring revenue, but getting started, of course. On Kickstarter, there seems to be mostly either physical objects or games. I managed to find Light Table which was funded in 2014 to the tune of  316,720$ by 7,317 people. Checking the repository, there seems to be non activity from the beginning of the year.