[CSDb] - User Forums

You are not logged in - nap

CSDb User Forums

Forums > CSDb Info > CSDb V2

2005-11-09 12:51

Perff
Administrator

Posts: 1665

CSDb V2

As some of you might have seen, there have been talked a bit about a CSDb V2.
I have also had this idea for some years now, because of some basic flaws in the originally database design of CSDb.
Now is the time to try to do something about it.

But we, the current CSDb staff, don't have the resources to do this, so we need your help!

Here is what we need:
1. Somewhere to host CSDb V2. The current hosting for CSDb is not very good, as some of you might agree on, and it would be very difficult for more people to develop on it, so a new hosting place is needed.
The requirements for such place are not to much. Talking about performance, CSDb can run on any normal computer today with no problem, and a few gig of disk space should be ok. The daily trafic sums up to about 40-50.000 hits and 300-400mb.
Some kind of CVS or similar to make it easy for multiple people to develop on it, is also required. (PS. We got no money! )
2. People that would help. First of all we need some willing developers - PHP & SQL. I don't think we should get too many - 2-3, perhaps 4 (plus myself :) ) We'll figure that out.
Also some design people, to make the site look kewl. :) And finally some people with deep knowledge of the scene so we are sure to get around every corner in the design phase.
3. Time and patience. :)

The plan is then to put togehter a CSDb V2 team (not to big, 6-8 people tops), and figure out exatcly how to make it. Of course we should look at the current CSDb, but I sugest we make everything from scratch.
Most importaint is the design phase. We must try to take all into account when designing it. I think this is the most importaint part of it all.
Then it gets coded, designed etc, and when it is ready for release, we transfer all the data from CSDb to CSDb V2. :)

Even though the CSDb team is the ones who should make it, it dosn't mean that the rest of you have nothing to say. Perhaps we could post our plans somewhere for all of you to comment on.

That is roughly how I see it.
Now lets have a nice little discussion. :)

... 91 posts hidden. Click here to view all posts....

2022-05-27 15:25

Oswald

Registered: Apr 2002
Posts: 5017

google tells me there are better ways to represent uncertain dates

https://stackoverflow.com/questions/27726716/how-to-represent-d..

date should be just date, and then another field describing the fuzziness.

2022-05-27 16:05

spider-j

Registered: Oct 2004
Posts: 444

I agree with Oswald. Date in a database should be date / unix timestamp. If you don't need all details of it in the frontend: don't use it. But if needed later on you will always have the flexibilty to extend it without changing the database structure.

2022-05-30 19:31

Raf

Registered: Nov 2003
Posts: 343

Quote: How likely is it that the source code of current CSDb will ever be made public? Same with the underlying database. I have a feeling the answer would be "you can pry it from my dead, cold hands". But I'd like to wrong on that one ;)

As for starting from scratch: sometimes that's the way to go, (try to) implement things properly from the start. From what I've heard the current code is hack upon hack upon hack.

True, from my playground: back in time (10+ years) there was custom coded forum on c64power.com which I've tried to make better but finally I ended up exporting entire database using my own written tools to MINIBB forum engine and when it was like they charged money for many things that were available right from the start on other solutions entire forum was auto-converted using someone's else script to SMF (and it's still functioning at http://c64power.com/forumng/ [ng was meant for new generation ;) ])

Actually it's better to define new (proper and extensible) database right from the start with decent MVC code and convert existing data than dig in knee deep hacks which arose during 20 years.

2022-05-31 00:50

TheRyk

Registered: Mar 2009
Posts: 2061

Quoting spider-j

"make it mobile friendly". This is really "simple" front-end stuff which could be achieved without touching any of the discussions which structure the database should have.
...

I've seen database stuff turned mobile-friendly ending in some front-end that might work on mobiles but is hell to use on real hardware (here meaning browsers running on computers, not smartphones). And once enough hipsters fall in love with that shiny new stuff, the old stuff ain't working anymore on old frontend and no-one cares to update it anyway as it's old...

No, thanks, but I'm with Frantic here :)
#donttouchmycsdb
The few things I'd like to change wouldn't need re-inventing wheels or starting from scratch but rather turn the wheel back when it comes to certain whacko (sub)categories or flags
But this is only me - grumpy old Ryk, not an official Mod statement

2022-05-31 16:03

Monte Carlos

Registered: Jun 2004
Posts: 351

I see that the csdb is in it's 20th year and as we know from old discs there will be times when everything falls apart. This is not the case currently, however I can understand the maintainers doubts about keeping csdb V1 until it is too late. I think it is very foreseeing to work on csdb V2 as long as csdb V1 is still in place. Nobody wants to shut down csdb V1, instantly.

2022-05-31 18:11

instant

Registered: Mar 2020
Posts: 20

Quote: I see that the csdb is in it's 20th year and as we know from old discs there will be times when everything falls apart. This is not the case currently, however I can understand the maintainers doubts about keeping csdb V1 until it is too late. I think it is very foreseeing to work on csdb V2 as long as csdb V1 is still in place. Nobody wants to shut down csdb V1, instantly.

I figured it would run concurrently to get everything up to par. It would sync every hour or so to stay up to date with newest entries.

Later when everything is solid and tested we can turn on the ability to submit new entries. Even then CSDbv1 would still operate as usual but it would not get any updates that were made only at CSDbv2.

If it takes a few months to work out the details and get things going that still might be considered an "instant" in contrast to the time of the first post in this thread. :)

2022-05-31 21:01

bOOZElEE

Registered: Dec 2002
Posts: 34

All credit to your enthusiasm, but I think you're underestimating the complexity a bit. very. ;)
But don't get me wrong or even let my words lead to slowing down your eagerness.

Some words about tech-stack (despite the usual "concept first, tech-decision second"):
In my eyes considering the approach of implementing CSDb V2 as appwrite/nodejs/nosql clutter is very questionable: not only does a new (rotten) JavaScript framework (react, vue, whatever) come out every few years, which is then hyped as the new hot shit (pardon my french), but also node itself is a filthy juggernaut: you have such a dependency hell, countless packages are pulled/installed, some (if not most) of those no one ever looks at or let alone review their source-code. (Just pops up in my mind: in 01/2022 the node libs 'color' and 'faker' were on purpose rendered useless by the author resulting in thousands of production systems being broken 'relying' on these libs). No one wants to maintain this.

CSDb V2 is not intended as another short-lived customer fancy-app project.

If CSDb V2 should be up and running for the next 20 years, drop the idea of using nodejs-crap. Well, everything that comes after 20 years won't interest most of us anyways, but nevertheless someone wants to/should maintain the stuff up to then or maybe for a few years more ...

Regarding database: due to the necessary relations, nosql imho would be bad choice. Better use a 'real' database like postgresql, mariadb or something like that. But some other people pointed this out already.

As there are a lot (and by lot I mean a lot) of assets aka releases of all different types available (download, tagging, etc.), also a proper File Abstraction Layer (FAL) is necessary. This needs some brainz. Huge brainz.

Even as the db-scheme is not yet ready, nevertheless I'll just throw in the following as a possible tech-stack
(In hope of earning another +1 for 'Tech stack first, concept second.' chrchr):

Haskell in combination with Yesod

"Haskell is a powerful, fast, type-safe, functional programming language."

Granted the learning curve is steep and hence many people don't work with it, but it's miles apart compared to php-fumbling or even js/nodejs-crap and once you're "in it" you will appreciate the benefits.
Just my personal opinion though, but Haskell should be worth evaluating when time has come to decide on tech-stack.

Just my two Złoty

2022-05-31 23:45

Skate

Registered: Jul 2003
Posts: 490

I once offered help for CSDb V2 and got declined. I'm fine with it. But reading some of the discussions, i'm glad i got declined. :) Pleasing a large community is always a difficult task.

About V2 db being sql/nosql, why not both (if required)? I use sql/nosql side by side. Almost every system have some audit logs, sessions etc. Nosql is very suitable for these tasks. About rest of the complex query requirements, relational databases save a lot of time and have good performence.

I agree with bOOZElEE on not necessarily using the state of the art technologies which would be a shame to use in a few years according to newbies. :) But i disagree with "nodejs-crap" kinda words. They all have their place. For instance i use PHP for more than 20 years now. But when a realtime connection (Web Sockets) required, i don't try to solve it using PHP. Node.js is very suitable for the task.

Of course you can use anything, PHP, node.js etc are not the real subject here. But most of the time people choose either one of them. I use their powerful sides together. It applies to backend, frontend, database, everything...

2022-06-03 05:11

instant

Registered: Mar 2020
Posts: 20

Ok... I've staged an Apollo GraphQL server with my first pass at building the required queries and resolvers. This is read only at the moment. It may be missing some data and other items might need to be renamed or rearranged to make more sense. The goal is to be able to pull exactly what is needed for any page.

https://csdb.idolpx.com

Please play with it and leave me some feedback here or hit me up in IRC. I believe I built things in a way that we can get at any data across all relationships. To avoid duplicating data, anytime a single ID or array of IDs for a given entity (Releases, Groups, Sceners, Handles, Events, BBSs, SIDs) is referenced it loads a single unique copy of that data and adds it to a base array in the query response. So in instances where a single handle or group is referenced multiple times for multiple things, only the ID is listed in the nested data. The detail can then be looked up by ID in the base array.

All of this is done using only JSON documents. No database yet.

Next I will start working on the mutations. That is going to be interesting.

2022-06-10 18:57

Count Zero

Registered: Jan 2003
Posts: 1821

Quote:

https://csdb.idolpx.com

borked after a week already or what is this?
But since the original topic and intention here is hardly being met anhow - closing to avoid further resurrection.

Previous - 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 - Next

Refresh

Subscribe to this thread: