How to avoid fearing data migration?
So I've been spinning my wheels on multiple side projects lately due to one, main concern: Data/schema migration. My projects are often applications to store data about my life. Anything from wiki pages, to file storage, it's just side projects that tend to interest me. However lately, I've been largely obsessing with choosing "perfect" schemas in projects that are more than just inside of a DB. If it was in a DB (MySQL/etc), migration isn't a concern, it's well understood. However I'm often writing my own storage mechanisms, such as storing images on disk or chunked bytes, whatever. So then if I realize I'm missing something, say crypto signing the blobs, I fear needing to change them after I've uploaded a TB worth of data to them. So I don't even know what to ask, but I figure this has to be a more common problem than I realize. Are there articles that talk about managing your own data and ensuring that it always has a migration path? Hopefully my problem is clear, but I'm sorry for not being able to define a better question in this context. Any tips for similar projects would be appreciated! 0 comments on Hacker News.
So I've been spinning my wheels on multiple side projects lately due to one, main concern: Data/schema migration. My projects are often applications to store data about my life. Anything from wiki pages, to file storage, it's just side projects that tend to interest me. However lately, I've been largely obsessing with choosing "perfect" schemas in projects that are more than just inside of a DB. If it was in a DB (MySQL/etc), migration isn't a concern, it's well understood. However I'm often writing my own storage mechanisms, such as storing images on disk or chunked bytes, whatever. So then if I realize I'm missing something, say crypto signing the blobs, I fear needing to change them after I've uploaded a TB worth of data to them. So I don't even know what to ask, but I figure this has to be a more common problem than I realize. Are there articles that talk about managing your own data and ensuring that it always has a migration path? Hopefully my problem is clear, but I'm sorry for not being able to define a better question in this context. Any tips for similar projects would be appreciated!
So I've been spinning my wheels on multiple side projects lately due to one, main concern: Data/schema migration. My projects are often applications to store data about my life. Anything from wiki pages, to file storage, it's just side projects that tend to interest me. However lately, I've been largely obsessing with choosing "perfect" schemas in projects that are more than just inside of a DB. If it was in a DB (MySQL/etc), migration isn't a concern, it's well understood. However I'm often writing my own storage mechanisms, such as storing images on disk or chunked bytes, whatever. So then if I realize I'm missing something, say crypto signing the blobs, I fear needing to change them after I've uploaded a TB worth of data to them. So I don't even know what to ask, but I figure this has to be a more common problem than I realize. Are there articles that talk about managing your own data and ensuring that it always has a migration path? Hopefully my problem is clear, but I'm sorry for not being able to define a better question in this context. Any tips for similar projects would be appreciated! 0 comments on Hacker News.
So I've been spinning my wheels on multiple side projects lately due to one, main concern: Data/schema migration. My projects are often applications to store data about my life. Anything from wiki pages, to file storage, it's just side projects that tend to interest me. However lately, I've been largely obsessing with choosing "perfect" schemas in projects that are more than just inside of a DB. If it was in a DB (MySQL/etc), migration isn't a concern, it's well understood. However I'm often writing my own storage mechanisms, such as storing images on disk or chunked bytes, whatever. So then if I realize I'm missing something, say crypto signing the blobs, I fear needing to change them after I've uploaded a TB worth of data to them. So I don't even know what to ask, but I figure this has to be a more common problem than I realize. Are there articles that talk about managing your own data and ensuring that it always has a migration path? Hopefully my problem is clear, but I'm sorry for not being able to define a better question in this context. Any tips for similar projects would be appreciated!
Hacker News story: How to avoid fearing data migration?
![Hacker News story: How to avoid fearing data migration?]()
Reviewed by Tha Kur
on
July 02, 2018
Rating:
No comments: