This is still very much early-days, but I’m thinking about building a distributed link aggregator. Some notes:

  • no servers (all P2P) except to connect peers
  • no “instances” so everything is in one namespace (so like reddit /r/community, not lemmy instance/c/community)
  • everyone is a moderator - moderation based on a web of trust type system
  • desktop first, mobile/web later - web would require servers, mobile may have platform-specific issues
  • no plans to integrate with fediverse, but could be possible
  • initial intent is to be text-only - file storage could be supported later in an opt-in basis (for now, just link to an external img host), but I’m worried about disk storage since there’s no central data lake and everyone needs to chip in some storage space

Some technical details:

  • written in Rust and ReactJS - Tauri-based
  • uses Iroh for data synchronization
  • looking into Appleseed and maybe TrustNet for moderation

Current status:

  • rewrote Appleseed and part of TrustNet to Rust for perf testing last weekend (Rust is waaay faster) - will probably rewrite again once I finish reading the paper (current code is a direct port)
  • have basic Tauri+Iroh app, but it’s a glorified chat app w/ no syncing between peers (a peer can join a chat, but will lose access if the host goes down)
  • reading source code for SimpleX (re-learning Haskell too) - could be interesting for DMs or something

I’m not sure how long I’ll work on this, but I’ll probably release some libraries under a relatively liberal license, so something like LGPL at the most stringent, but probably Apache/MIT (depends on if I need to clean-room the appleseed implementation).

So, if there’s interest here, I’m happy to post updates for discussion periodically.

  • @sugar_in_your_teaOP
    link
    English
    21 month ago

    simple website

    WebRTC might be sufficient, just like it is for libp2p.

    My app is all in React using Tauri, so I would just need to replace a few calls to a local library with webRTC support, which should be pretty easy.

    phone running a background service

    That would be 100% opt-in. I would even allow disabling sync over data, so you could still post and whatnot offline, and it would sync with peers later.

    But that’s also why I’m doing desktop first, I don’t need you worry about limitations while I’m prototyping it.

    peers need to be online for their data to be accessed

    I intend to have enough duplication that it shouldn’t be an issue.

    To start, I also intend to have always online “clients” that, combined, have “all” (as much as any p2p app can) of the data until enough people use it to have it self-host.

    apply it to greatly improving products that already have traction

    That’s usually what I do anyway. I really don’t want to head a project, but, as you mentioned, other stuff in this space seems to have failed:

    • IPFS - too slow, Iroh improves that
    • Hyphanet (OG freenet) - focused on “web pages” and anonymity, which attracted the wrong people
    • new freenet - isn’t available yet (soon!!) - once it is, I’ll play with it, but it’s libp2p-based, so probably slow
    • scuttlebutt - only small groups, I’m looking for a Reddit alt
    • aether - ephemeral in nature (6 months), and the project seems dead (no activity for 3-4 years)
    • cryptocurrency-related - almost everything else, which it attracts the wrong crowd

    I’ve contributed a bit to Lemmy, but I think lemmy is fundamentally flawed so we’ll need an alternative eventually. I hope to make a bridge to act like an instance eventually, which would at least allow a federation of p2p networks (could be cool).

    ways to make moderation better

    Better is subjective, but yeah, I’ll definitely be pulling from Lemmy datasets for simulations. However, it would probably need to be a browser extension due to the data and interaction I want.

    There are three types of trust:

    • trusted - I have proactively approved something (most web of trust systems)
    • untrusted - no productivity whatsoever
    • distrusted - I have proactively flagged something as harmful

    Lemmy mods handle the third and user votes kind of handle the first two, but if you don’t trust the mods, you’re SOL. It’s also at risk of Sybil attacks (e.g. vote brigading), so it’ll have similar issues as Reddit.

    I think I can all three types of trust with two inputs: flags and votes. Explicit trust of a user could be helpful (e.g. bots or community mods), but hopefully not required. If it significantly improves performance, I might add a “mod queue” so users can train the moderation system (e.g. do you agree with this moderation decision?).

    I think it’s an interesting area to explore, and I don’t see anything using it, which means it either:

    • doesn’t work - so I shouldn’t propose it to lemmy
    • is hard - so experimentation is valuable
    • is too expensive - probably the case (initial naïve simulation with 100k nodes took ~3s to initialize, checking is fast)

    I’m hoping people think it’s harder than it is, which is why it’s overlooked. Votes and blessed moderation are easy to implement, so I’m banking on that. There may also be a way to integrate some limited machine learning and get hardware acceleration from a GPU or AI chip (e.g. Google Pixel’s AI chip), so “expensive” may be less of an issue once I go public.

    Anyway, thanks for the thoughts! I may need to discuss web support with Iroh devs. I know they have a web demo, but I think it routes through a web server, not an actual p2p web app, but I haven’t checked.

    • Lung
      link
      fedilink
      English
      21 month ago

      Cheers bro, great to meet you and I’m sure we would get along. Hoping for your success and joy