Over Engineered

00:07.10
Chris Morrell
All right? Welcome back to overengineered the podcast where we asked the question. What's the absolute best way to do things we already have a perfectly acceptable solution for ah today I'm here with Ian Landsman hey Ian you want to say hi I'm good. Um.

00:18.30
Ian Landsman
Hey how are you.

00:24.33
Chris Morrell
I Ah I reached out to you a little while ago because I just thought it would be fun to talk I didn't really have a game plan but I just I was interested in the fact that you and I have both ah been working on the same applications for a really long time. Um, and. When we started talking we we thought well let's just ah, let's pick a topic and see where it goes and and we started thinking about caching because I know you're in the the ah I know would you say it's still the very early stages are you further along in this rewrite? yeah.

00:57.44
Ian Landsman
I would say early stages. Still yeah.

01:02.84
Chris Morrell
But it sounds like you're already kind of thinking about these optimizations and it might just be sort of a fun thing to explore. Um, but ah I don't know I say this every time but I'm going to say it again. I I can't imagine that anyone who is listening to this podcast doesn't know who you are but do you want to just do ah. Ah, quick intro just in case.

01:23.47
Ian Landsman
Yeah, so I founded a company called Usercape ah, twenty years ago our main products a product called helpspot which is a customer support application help desk tool and bootstrapped and profitable for every day of. 20 years almost here and yeah in terms of like the internet community. You know, been involved in larebell since very early Taylor worked at Usercape for almost three years um run the first few laracons run laracon line run lara jobs so that's kind of my open source community stuff end of things and then help spot is sort of the day in day out business ah product and yeah, that's kind of I'd still of code still like the code that's kind of what got me into it to begin with definitely not. Seeing some of the names you have in this list I am not the level of coder of those people for the most part but I'm a serviceable coder I'm a practical coder which I think our topic is kind of good at because I feel like it's one of those sort of like real world things that you hit burst just like.

02:32.50
Chris Morrell
Right.

02:35.12
Ian Landsman
Architecture Astronaut type stuff. This is like a real problem that real applications have so.

02:41.29
Chris Morrell
Yeah I mean and I think it's really interesting I I don't meet many people who are you know as established founders who are still in the day today weeds of coding like ah you and I both are so it's it's cool to see that I mean. Was that something that you was that something that you did really intentionally um or has it just kind of worked out that way.

03:04.72
Ian Landsman
Yeah, it's um, somewhat it's sort of a combination of a whole bunch of different factors but basically like definitely in the beginning I was the only programmer and then that was that and I built it and all those things and it's kind of ebbed and flowed like there's been periods where we had like. 2 or even 3 programmers and those were periods where we have less. Um, so there's been periods where I'm not doing any programming like there's been years where I did nothing nothing in terms of programming. Um, but then recently I've gotten back into it I kind of feel like. I mean first of all I just like it I've been keeping up on it the whole time. But even if I'm not actively always been doing code in help spot and some of our other things I've been building other layerbel related sites or side projects and things like that. So I'm up to speed on everything and then I do like doing it and then when it came to kind of this. Um. Next generation help spot. We've started working on it just felt like we've actually tried to do this twice before and it's failed both times before um and I think at least in part and entirely my fault so one ah hundred percent my fault all the failures. And I just felt like god to do it. It's just I'm just not the type of person who's going to be able to communicate it to other people is what it comes down to well and so um, you know I'm going to get in there and I'm going to do like the bays one of this and then.

04:23.32
Chris Morrell
Sure No I get that.

04:33.79
Ian Landsman
Um, you know from there once like the core vision is established and a lot of the core structures are in place I feel like then it's a lot easier to be like Okay, we're going to spec out this feature. We're going to spec out this ui thing like we could We could do that? Um, but just laying out the whole groundwork and. s just too many things that are interrelated that I don't think very well in terms of like oh let me prethink this whole thing out and every implication and what the Ui is going to be and then write that all up and commit. So like I need to be in there feel in the code feel in how it looks in the Ui like. The actual interactions and things and then be like oh yeah, this makes sense or this doesn't and so that's what we're going to do and we're gonna see how that goes.

05:14.60
Chris Morrell
Yeah, no I get it I I think that it it is really true, especially like I don't know you just you you end up. Um, just having so much context in your head over the years that you don't even realize it's there anymore you know.

05:27.51
Ian Landsman
Yeah, yeah, and and there's obviously you know thousands of people who work at Microsoft and thousands of people who work at Apple who.

05:33.40
Chris Morrell
And so I get that it's hard to communicate some of that stuff.

05:43.50
Ian Landsman
Are very good at that type of thing right? They never write any code and they're writing documents and specs for whatever software they're doing and goes through approvals and like there's all kinds of stuff people with different type of talent. Um, but that's that's not me so um, going to keep it pure.

05:48.59
Chris Morrell
Sure.

05:59.30
Chris Morrell
Um, it it Also I mean I think it also really depends on how your company and your team is structured right? They're built to do that. They have so many many systems in place. Yeah, exactly.

06:05.26
Ian Landsman
Sure you just have to be It's a whole different thing like there's schedules and there's trillions of dollars involved everything else. This is my little company I can do whatever I want with it right? and so it can be whatever I want it to be which is also part of it like I just like doing this and. I want to be in there doing it and so I am.

06:27.80
Chris Morrell
Yeah, well you know we've been. We've been ah running since about Ninety Ninety seven and you know for ah I would say about the past.

06:34.14
Ian Landsman
Ask.

06:42.40
Chris Morrell
Eight or nine years we've been migrating to laravell. We just kind of decided um rather than trying to sort of set a timeline for getting that done. We've just moved functionality over as as it made sense. Um and just kind of manage everything through the ah the load balancing side of things just like.

06:44.00
Ian Landsman
Right.

07:00.59
Ian Landsman
And.

07:02.18
Chris Morrell
Path by path essentially um and that's been a really interesting way to rewrite an application. Um, but I think you know obviously the appeal of just like sitting down and writing the whole thing again from scratch is is very real.

07:06.83
Ian Landsman
That is fascinating.

07:17.74
Ian Landsman
Sure.

07:19.88
Chris Morrell
Um, and I I won't I won't pretend that I haven't you know, kind of gone down that path a few times over the years too and and either abandoned it or or you know I don't know how how the codebase over there looks like but there's lots of like well this class is named like. Legacy because there was a new one introduced that was supposed to supersede it. But then that migration never fully happens. So now we just have the legacy one and the modern ones sitting next to each other and you just have to know which is which all that all that nightmare. Yeah yeah.

07:47.20
Ian Landsman
Right? Yeah, a lot of real world stuff in there? Yeah yeah, we did something similar with helpspo where we moved it on to label because ah it's obviously pre.

07:56.46
Chris Morrell
Um.

08:01.80
Chris Morrell
Right? right.

08:02.27
Ian Landsman
Like pre any framework really existing m ph b as your app I'm sure is and so yeah, so we moved on ah to Labelle and then you know over time. It's definitely not like you wouldn't look at it and think it's a label app.

08:18.16
Ian Landsman
Fully because there's definitely odd folders and function libraries that are thousands of lines long and things like that that are not in any way how you do it now. But ah, but yeah I do think rebuilding from scratch I mean obviously it's old hat now people are basically you know.

08:19.21
Chris Morrell
Aha.

08:35.86
Ian Landsman
Consensus is kind of that you shouldn't do it and all those things and I tend to agree I think that for me what kind of put me back over the edge was like we had some business model changes. We want to implement as part of this so then to me that made a lot of difference. It's like all right if we're It's not just like it's going to be new and nicer to work on but it's Like. We can't do the business model that I think makes sense going forward on the current structure and so then that's like all right, That's like an actual real reason to to do this? Yeah right, right.

09:04.18
Chris Morrell
Right? You're building a new product. It's it's with the you know it's with the history of the existing product but I get that sense? Yeah, um, but but another interesting thing about that process is you're coming to it with all the experience of running.

09:13.18
Ian Landsman
Yeah.

09:20.90
Ian Landsman
Out.

09:21.97
Chris Morrell
You know running ah an application for for 20 years um and you know I I remember I can't remember what episode of ah, mostly technical. You guys were talking about it but the whole um Dynabo Db Access Pattern question like needing to understand your access patterns.

09:38.17
Ian Landsman
Yep yep.

09:39.79
Chris Morrell
Like you know you are in that position where you actually do kind of already understand a lot of this stuff in a way that a a new application really wouldn't um and so I imagine that's why you feel comfortable kind of like.

09:49.48
Ian Landsman
Yep.

09:55.88
Chris Morrell
Ah, taking on a bunch of caching considerations really early rather than you know I think traditionally you would piece that stuff on later as you see the need um but like so what? what has that been like it sounds like that's kind of ah just been something that you've been really.

09:57.97
Ian Landsman
Yeah. Um, yep.

10:13.58
Chris Morrell
Experimenting with.

10:14.79
Ian Landsman
Yeah, so that's exactly right I feel like it's probably not the place you should overengineer generally if you're starting a new app. Um, but we have you know some unique variations here because there will be existing customers moving on to the new.

10:19.53
Chris Morrell
But.

10:29.41
Ian Landsman
Platform pretty quickly. Um, ah so that right there there's going to be more load than you might have just if you were just obviously building a new app and getting your first customer and things like that and then um and then yeah I know the access patterns really well and even though the new platform will have some differences.

10:37.11
Chris Morrell
Sure.

10:49.11
Ian Landsman
Ah, ultimately like it's going to be very similar. Um, and so the places where it's heavy I know those very well and so so yeah, so that's kind of led me down to like which is you know into going to caching more initially and understanding where we can cut some corners and what things can be cached and don't really matter if. They are have a slight delay to them or what things can be cached because they're easy to clear the cache or all those kind of things. Um and go pretty deep on that initially more so than you might otherwise recommend if you are building a brand new app and don't really know what's going to be ugly and not. But yeah, the help desk. Space just has a lot of complicated queries in it because people want to see their helped us tickets in all sorts of weird and interesting ways. So ah, they want to you know, have string searches for a word and but then they're also filtering out.

11:35.19
Chris Morrell
Sure.

11:44.19
Ian Landsman
You know these other twenty conditions and there's a group condition and like there's all this stuff in there and the load can definitely be quite intense So That's like a multi-layer Thing. We're kind of working on from like the database architecture aspects and what we could do differently there. Um. And then also ah then the caching layer of what can we cache and only run once in a while and have it be okay and um, so yeah I mean there's there's a lot there but ah.

12:17.29
Chris Morrell
What's what's the. Ah, what's the thing that um, what's the what's the thing that you've done so far that you ah would would be most nervous showing someone else. What's what's the what's the most worrisome caching. Ah.

12:27.18
Ian Landsman
Yeah, so I'm working out that's a good good starting point. Yeah, so well this is so this is like a toll overngineering thing right? So there's we're going to most likely be using Redis to cash redis obviously extremely fast.

12:33.50
Chris Morrell
Cash and decision. You made.

12:43.70
Chris Morrell
Yep.

12:44.12
Ian Landsman
And but it just bothers me to go to the cache like okay so the standard way to do. It would be like okay like here's a query we're going to cache this query and wrap it in a you know larabel cache or whatever. Um, and we have like might have 10 of those in 1 request that our cache let's just say. And then but then each one of those is like a network request and even though it's to a fast redis server. It's still Ten Network Network requests and then that's like I don't like that. It's like oh man, that's a lot of network requests right? So then it's like what can we group to get there. This is kind of the thing. I.

13:04.54
Chris Morrell
Right.

13:20.51
Ian Landsman
Been working on most recently is like what of these can I group together conceptually so I have these like cache groups of like the tenant is like 1 cache group and the user has a cache group. Um, and it's like okay for those. Ah, we're just going to go make the one network request and request all the keys. And get it all back at once and so then maybe now instead of 10 network requests. You know you're always going to have the tenant one probably and you're probably always going to have user one and maybe there's going to be like 1 other depending on what module the app you're in but maybe now instead of 10 there's 3 and so I like that that's like okay like. That's that's much better than 10 um, and it's probably like literally whatever it's going to be like it is it ten milliseconds or is it maybe even less if it's a really fast connection or maybe it's slightly more? Um, but yeah, it's just an area where I'm like nah you know I'm in here I know that like every time I use these. Ah, like the tenant cached items like I need all those anyway a lot of the time. Um, and if I get an extra one once a while like it still doesn't matter because just the making the network connection is slower than like having it send back this you know one extra key or whatever and so. We're just going to try that out and see how it goes so so far it works but um, it hasn't been you know tested heavily yet. So we'll see if it holds up. But so far it's it's looking pretty good.

14:47.28
Chris Morrell
And so then are you are you just kind of leaning into um, sort of accepting occasional eventual consistency or are you also really thinking through Cash invalidation strategies heavily.

15:01.81
Ian Landsman
Yeah, so pretty heavy on the cache and validation I mean the thing with a help desk app is there is a ton of settings and they don't change that often and when they do change. You know it's very controlled how they change So um, for some things.

15:09.00
Chris Morrell
Right.

15:19.33
Ian Landsman
Ca invalations like super straightforward. Um I do have like kind of a Baseline Cash invalidation setup where like the key of the cache includes like the updated stamp of the tenant we're talking about the tenant. Um, so you know I could just like touch that and that'll invalidate the whole cache and then obviously it'll.

15:30.34
Chris Morrell
And.

15:38.95
Ian Landsman
Be fresh on the next run. Um, but there are places where we're just going to actively remove you know reset the cache and delete the key because again if you're just like changing a setting that gets changed I mean it might be years between changes or whatever like um, we can just go ahead and delete that and it'll be fresh again for the next one.

15:54.37
Chris Morrell
Sure.

15:58.87
Ian Landsman
But yeah, did the this group caching structure does have some some ah key manager. Basically that creates the keys and it does part of that include the updated stamps that we can kind of quickly do that. Ah, 1 other thing. That's built in there is that the tenant updated at is actually in every key so that if we want to clear the whole tenant because like whatever customers having a problem and we just want to like be able to clear all the cash every cache that's for the tenant.

16:21.30
Chris Morrell
Okay.

16:33.49
Ian Landsman
Then we could just update that and it's going to clear actually all the all the cache keys. So um, which we'll see I mean that has its own like downsides potentially right? Like if you have a very large customer like maybe don't want to cache all or clear all the ah caches simultaneously. But I think. Again, this is what where it's nice, knowing our use cases very well that like any 1 tenant's not actually going to be that big a deal to have like their caches cleared all at once and be cold. So I think I think it'll be fine, but um, but yeah, so that's.

16:57.10
Chris Morrell
Right.

17:03.49
Chris Morrell
Um, I mean that's something that is something that I've been leaning into actually more is just I think um I used to try to be really really thoughtful about you know Using. Um. You know, clearing as little of the cache as possible when something changes and I have found that it's just so much easier to just be like I mean I'm almost just like blow up the whole cache when anything changes. Yeah because like again, it's just a performance thing. So The app is still going to work just fine.

17:19.46
Ian Landsman
Right.

17:29.86
Ian Landsman
Scorched scorched earth right? yeah.

17:37.50
Chris Morrell
You know the next request for a handful of people is going to be a little bit slower as as some data has to get loaded but it ends up being pretty minimal I guess it depends on how how heavily you're relying on caching. But um I've just found that the over under on that is it's just so much nicer.

17:37.63
Ian Landsman
Yep.

17:49.29
Ian Landsman
Yeah, it.

17:56.71
Chris Morrell
To have that assurance then hunting down these like okay well this update this model was updating and like it wasn't touching its parent and so then I need to like you know all that nonsense. It's just like just wipe the whole thing you know? yeah.

18:06.16
Ian Landsman
Yeah, you get into all that yep, blow it away? Yep Yeah I agree That's why I kind of had that option in there for everything is like up. We could just blow this away easily and not like worry about it if there's some we aren't anywhere Yet. We're like. A change I'd feel like well we should just always do that. But I'm sure we'll get there because there are some very complicated aspects where it's like well in this case, we're just going to like blow everything away and obviously we could just do it as admins too. But um, even just as part of the regular flow of the app. It's like okay this is like touches too many places. Let's just like clean up. A bit. Ah, but yeah I mean there's all so kinds of like um you know initially helps I had no caching or you know current version of helpspot and or effectively no caching very little caching. There's an On-premise app initially um and it's still available On-premise and so we didn't hit it too much early on because it's like.

19:01.61
Chris Morrell
Sure.

19:01.63
Ian Landsman
Everybody's putting it on their own servers and generally it's fine like unless they had a huge installation and even then they'd have a bigger server. So really like you could kind of run any queries and it all kind of worked okay because it was just the 1 tenant on its own server. Um.

19:11.40
Chris Morrell
Yeah.

19:17.20
Ian Landsman
But then has like our cloud stuff got bigger. It's like oh all these crazy queries people are running and coming from this On-premise world. We'd like let people do anything like you could literally write your own in the f in the filtering of hubspot like you can literally put your own sql in there. It's like so people just can literally do anything and then the the builder has like no constraints like.

19:24.15
Chris Morrell
Right? right.

19:32.26
Chris Morrell
Right.

19:36.87
Ian Landsman
You know the database schema wasn't necessarily optimized for a cloud type deployment. So there's a lot of heavy heavy queries and things so that started to really be on the database server. Once we cloudified it all. Um and it's not.

19:41.11
Chris Morrell
Sure yeah.

19:52.21
Ian Landsman
Just explain that quickly. It's not a regular like multitenant Saas app. It's a single database server and every customer gets actually a tiny aws. Um, ah Bm And so.

20:03.96
Chris Morrell
Um.

20:05.13
Ian Landsman
We just deploy it like a million little on-premise installs that do connect though to 1 single database server so that database server can get heavy but each customer has its own database on the database server. Um, so but still obviously then one customer can impact another even if they're on their own databases because.

20:14.58
Chris Morrell
First sure.

20:23.56
Ian Landsman
1 server. So ah, so we did a bunch of caching stuff in helps spot current as much as we could um and one of the things. Ah my big takeaways from that and something I'm using in the next generation helpspot is. It's super beneficial even though just to do tiny caches like. Like we're only caches for 30 seconds or something like that because when you have so many customers simultaneously like 30 seconds actually huge like you're saving thousands of queries if you just have like a thirty second cache in this one area or whatever. So um, so that's another thing that factors in for me because it's like we're small like.

20:45.80
Chris Morrell
Um, yeah.

21:02.12
Ian Landsman
We don't have impotent money for our you know databases and things like that and the database is obviously the most expensive thing to throw money at if you need to throw money at something so it's like hey if we can keep on smaller databases. Ah the longer we can do that. That's super beneficial because going up to that next. Tier of databases always at least twice as much and gets expensive pretty quickly. So So yeah, so there's a lot of stuff like that too when we get into like the ticketing views and like the filters and the counts and things like that where it's like even if it's just this.

21:21.17
Chris Morrell
Yeah, yes.

21:38.51
Ian Landsman
30 seconds that seems like kind of nothing. It's actually can be a lot because if you have you know 50 agents. Um, and so the other thing with helped us is this the kind of tool where everybody's in it all day like it's your job to literally be in this tool all day. So if you have 50 agents.

21:50.98
Chris Morrell
Um, yeah.

21:56.55
Ian Landsman
Like they're in there loading pages just constantly and move it around the system and everywhere they go to. There's a count of how many tickets are assigned to them and how many tickets in this queue and whatever all the different counts. Um, and so you know just those little caches where yeah, you don't want it to be. Ever delayed too long because people are making decisions based on the counts and things so they want to know when something pops in a queue or how many are in there but you can still shave a lot of queries off even with these short caches and then there's other areas where it's not so important like what is the difference between. 42 and 43 items in this queue potentially like there's probably not like nobody's looking at it and it's like oh I'm not gonna touch it if it's 42 but when it gets to 43 like then I'm gonna get in there and do something about it. So.

22:39.45
Chris Morrell
Um, sure.

22:48.67
Ian Landsman
So that's where then there's a little bit of wiggle room in other areas where it's like well for this one. Maybe it can be 5 minutes because but slightly delayed. It's not a huge deal so judgment calls there. We'll still have a thing where there's settings for customers that probably impact the caching in certain areas like There are a team that is like very real time then maybe it's gonna be a very very short cash but you know are the hub a way to do that or we'll have a way to do that for them or so i't that part we haven't got to yet. But I haven't totally figured that put out but something like that. What's that.

23:19.94
Chris Morrell
You'll you'll have their ttl cached as well. You'll have the the the cache ttl cached for them. Ah.

23:26.79
Ian Landsman
There right? right? Yeah, exactly. So ah yeah, so we'll see I mean in the current current help spot. There's actually a way for people to basically opt out a bit essentially on certain queues and have it just be actually real time I don't think we'll go that. I think will it will always be cached. But even if it's just like 15 seconds or whatever like they just we're gonna go to shave that off a little bit.

23:50.48
Chris Morrell
Right? Yeah I mean do you find because ah in my experience the closer to real-time you get the harder. It is to effectively cache things because um, your your users end up noticing those inconsistencies a lot quicker.

24:00.31
Ian Landsman
Right.

24:05.66
Ian Landsman
Right? right.

24:07.77
Chris Morrell
Um, you know how are you? So are you still with all of those even these thirty second or 5 minute um count caches are you still thinking about invalidating them or is it just like we're just going to live with the fact that it might be 30 seconds stale

24:22.12
Ian Landsman
Yeah, so for those in particular um, we're probably not going to be able to actively invalidate them because it's things where they're they're creating the query essentially through a builder and even though in the next generation helps spot. It's going to be a little bit more.

24:37.24
Ian Landsman
We're gonna have a little more guardrails on there than we do with the current help spot. It still would be difficult because you might have hundreds of these filters in a big installation potentially and so like to like take every ticket that comes in and figure out which of the filters. It actually applies to become its own like set of. Giant queries and things so probably not going be able do that. So yeah, so for those you know, but this again where we know it really? Well, it's like the inboxes can be might just have no no cache at all or like a very tiny cache and because that's like that's 1 people always are looking at like.

25:09.14
Chris Morrell
Yeah, yeah.

25:11.80
Ian Landsman
What's new in the inbox I'm trying to keep the inbox empty or near empty like how many are in there. What's in there like okay so that's heavy cash or not cash. Um, like what we call like your myqueue like the stuff assigned to you again is something like. Gets looked at a lot and it's super important like if something was assigned to you. You want to know about it pretty quickly and not have to um you know you don't want like a 5 minute delay on that one or whatever. So but then you have these other filters that get built that are like a manager builds a filter because they're keeping an eye on a certain type of issue.

25:42.61
Chris Morrell
Sure right.

25:43.90
Ian Landsman
And once a day they're looking at that right? and so for that it's like well that doesn't need to be real time at all and so um, but it's hard for us to know explicitly its use case also because sometimes you build a filter and you do want it to be real time So there is a lot of like yeah, it's trick is tricky where it's like.

25:57.23
Chris Morrell
Right.

26:02.60
Ian Landsman
That's where I think those like ah shorter Cash bands come into play because it's like hey if we can shave a minute great like it's gonna be not a big deal to the customer. Even if they're using it semi real time. Um, but and then if they're using it not real time at all and they're checking it once a day then that's great and we're so. Still saving a bunch of queries. Even if it's not as much as it could if we knew for sure that they were only looking at this once a day. Yeah, we could cache it longer but it's still um, you know the all the queries are being you know again since we know the use case going. The queries are much tighter than in the current generation help Spot. So The queries are also going to be very fast. So It's not a huge deal. There.

26:42.66
Chris Morrell
Something that could be really interesting is like um I don't know if you could use pulse but you might be able to take inspiration from pulse All the um you know the sampling and and sort of analytics efficiency stuff like would you ever consider. Um.

26:51.83
Ian Landsman
Um, yeah.

27:00.24
Chris Morrell
Essentially like having some of that especially like the query builder stuff having some sort of dynamic caching where you're essentially sampling how often any individual key is getting hit and like dynamically adjusting the cache depending.

27:17.62
Ian Landsman
So definitely possibly um in in the current generation of helpspot we do we track like the number of times each filters run whether it's a count run or a fully loading the filter where like you're actually seeing the tickets as opposed to just like the sidebar count and right now we don't use.

27:25.17
Chris Morrell
But.

27:34.42
Ian Landsman
Ah, well no, we actually do use it a little bit to dynamically update. Ah, how long something is cached so it can change that and then we also use it to create a screen for the end user so that they can have some tools for like debugging like why things are slow, especially like in the on-premise one. It's like well you have this built in that takes 4 minutes to run like that's.

27:38.78
Chris Morrell
Okay.

27:53.56
Ian Landsman
Why it's slow or whatever. So um, so yeah, definitely some stuff like that. Ah I don't know if we're gonna need it for this. We'll see kind of I'm I'm going a little different path with that sort of issue in next generation help spot. Um, between the like.

27:54.92
Chris Morrell
Hooray.

28:12.61
Ian Landsman
Heavy and precise caching on one end and then a fair amount of like de-normalization on the other so that the queries can like pretty much never need to join or very on the count type queries. They'll never need to join and um.

28:22.60
Chris Morrell
Um.

28:31.19
Ian Landsman
You know in a lot of places they just won't need to join in general. So I'm sort of approaching it more from that angle of can we lean on the database and have all the queries be super fast when we do need these queries and not have to worry too much about.

28:41.54
Chris Morrell
So.

28:47.17
Ian Landsman
Getting all the way down to that level of like really tracking every filter or heavy query and then dynamically updating and all that stuff like hopefully it can be a little simpler in that zone and then we make up for it with like yeah if we don't do go database a little more.. It's super Fast. It's a really easy query that the database is going to. You know, just be able to crush very efficiently as opposed to ah you know getting into big huge multi-table Joins and all the stuff which we do now where every filter query is like has you know 3 tables joined to it and um, you know.

29:14.85
Chris Morrell
Right? right.

29:23.23
Ian Landsman
It's a big. It's a big ugly query that does all kinds of stuff. It's It's a monster so trying to avoid that monster query scenario.

29:31.97
Chris Morrell
Um, I feel compelled to at least say sounds like a great use case for event sourcing.

29:37.59
Ian Landsman
I have I have thought about that a little bit with you guys and all the stuff you're doing I So I know nothing really about doing event sourcing So I don't know I be quite scared but the same time it does I did actually kind of have that sort of thought about like oh this kind of. I see where they're going with this and conceptually the little bit I've poked around on verbs and ah yeah, probably is.

30:00.76
Chris Morrell
Um, yeah I mean when when you start thinking about denormalization I think that that's like a good a good ah indicator that maybe event sourcing is ah is a useful tool to reach for just because um.

30:09.82
Ian Landsman
A.

30:16.46
Chris Morrell
You know one of the upsides you know it's kind of like a power and weakness is um, you know it lets you kind of decouple the the writes from the reads and really optimize the reads specifically to the Reid case and the write specifically the right case. Um. And so when you're dealing with like a lot of data that needs to be read in these like really weird ways. Um, it allows you to ah retroactively just start pumping your data into a new table to improve your querying without having to really change anything on the right side. Um, so yeah I mean.

30:52.71
Ian Landsman
So I have a question for you there? Yeah no, let's talk about that a little bit because um so one of the things I don't think was has been totally clear to me although I'm not I I mean I've literally just looked at it a little bit I haven't gone I'm installed it I haven't gone deep at all. But um, is it.

30:53.11
Chris Morrell
I Just had to get that in. Yeah, go ahead.

31:04.52
Chris Morrell
Um, sure.

31:09.57
Ian Landsman
Kind of thing where you kind of have to do the whole app with it or else. It gets weird or can you drop it into a part of the app or what's your what's your analysis there.

31:14.82
Chris Morrell
No I mean where you know we've been. We've been running a portion of our exam system. Um in an event source manner for probably 2 2 3 years now. Um, and that is running side by side with the rest of the app. Both the.

31:27.70
Ian Landsman
M.

31:34.50
Chris Morrell
Modern lairville app and the legacy app. Um, and it's actually I mean Daniel's talk at laircon u I think was mostly um, ah inspired by some of the work that we did on that because that was really how both of us got introduced to event sourcing when he was working at Innachi. Um, and it was kind of this amazing thing because ah when you when you have this sort of like write model and read model mindset and I don't want to get too much into the event sourcing terms because the whole point of verbs is that we're not thinking that way. but but um when you start thinking. Okay I'm just. Firing off events that happen over here and as long as I throw everything that I can possibly think of I can kind of do whatever I want with it on the other end and so what that let us do with the exam system is we only wanted to move over actually 1 exam. Um. The rest of our exams are still actually still run through the the legacy system and um, we have sort of the modern projectors that are writing to the modern ah exam tables. Um, and we also just have ah backwards compatible projectors.

32:31.93
Ian Landsman
E.

32:48.42
Chris Morrell
Are writing to the legacy tables and so you could so you can still to this day start an exam in the modern system and then switch halfway through to the legacy system and it just works because the data is just kind of like bidirectionally going into 2 different places. Um, and so it actually like really.

32:48.48
Ian Landsman
M.

33:01.95
Ian Landsman
He.

33:08.10
Chris Morrell
Lends itself to kind of just dropping into places that that it makes sense. Um because you really can kind of just work with the event system and and the rest of your system is kind of too too isolated but communicating. Um.

33:11.95
Ian Landsman
Yeah.

33:24.43
Ian Landsman
A.

33:27.38
Chris Morrell
Pieces You know that said I think if ah, you know if I were starting a new project and I knew that I was going to be using event sourcing I I would probably be tempted to use it more heavily just knowing the upsides. But um, yeah I think that there's ah, there's definitely a.

33:40.45
Ian Landsman
Right.

33:47.33
Chris Morrell
Ah, plenty use cases where you just kind of pepper it in where you need it.

33:49.31
Ian Landsman
Yeah, all right Ill to take a look take a closer look in here see if there's a new tool little arrow in the Quiver right already already rewrite I've already rewritten this twice like.

33:54.98
Chris Morrell
Yeah, there you go read rewrite rewrite number three.

34:05.88
Ian Landsman
Basically the new the new next generation healthpo is on its second version already because it was like kind of waiting for livewire three. But then that was going to take like a long time. So I kind of went down the react path and was like well maybe I'll do like aner react. Um.

34:17.99
Chris Morrell
Um.

34:20.73
Ian Landsman
Which I'd never done anything with react so that was like a whole side quest of learning react and all that stuff and then in the end it was like and know it's like just didn't There's some obviously very appealing things about react, but same time I don't.

34:22.70
Chris Morrell
Sure.

34:33.92
Chris Morrell
It's tough tough.

34:37.24
Ian Landsman
I don't really want to just be in the front end that much like essentially like I don't think that way about it and so then live wire 3 was coming up and I was like ah we're just gonna do live wire 3 That's what we're gonna do so yeah.

34:39.50
Chris Morrell
Yeah, yeah.

34:48.48
Chris Morrell
Um, it. Also it really depends on your team because we went. We went down that path too and we still we still have actually a lot of react code. Um, and I I feel personally pretty comfortable in that space. But the rest of my team really does not like that you know they kind of. Put up with it as much as they need to, but it's not anyone's like strong suit and so it just you know it just means that every time we have to touch something that's in react, it's like okay well I have to get my my hands dirty because I'm the only one who necessarily knows how how it works.

35:07.31
Ian Landsman
Right? Ah, at.

35:17.94
Ian Landsman
Yeah, get Chris in here. Yeah.

35:23.67
Chris Morrell
And so yeah, we we definitely had that same like you know because I I had that I had the yeah experience that everyone had to some degree a number of years ago just when node was fresh and there was just this like glimmer of well what if you could just write everything in one language.

35:43.13
Ian Landsman
I.

35:43.58
Chris Morrell
Wouldn't that be incredible right? And ah you know I got pretty I got pretty involved in you know, right? You know we wrote a couple of sort of our um ancillary tools and node and and and we used Ember We used react. Um, just kind of exploring those world Worlds and ah.

36:00.38
Ian Landsman
M.

36:03.20
Chris Morrell
Think that premise is still really intriguing. Um, but yeah in the end if you're not like fully committed it kind of it really does suck to to be like one foot in one foot out

36:17.11
Ian Landsman
Yeah, yeah, yeah, it was clear. That's not really going to work that like the one put in one put out. It's gonna be super hard. Um, so then it was like whether you just have to do the whole thing in react and I mean with like label backend or whatever which is fine. Um.

36:32.39
Ian Landsman
And it's just a lot of weird the the main draw for me as I've talked about I think I'm mostly technical is that like the component story. Just super amazing and like there's just nothing even close to it like have so much pre-built powerful components that are accessible and super well tested and there's just nothing anywhere else.

36:37.78
Chris Morrell
Yes.

36:51.61
Ian Landsman
Literally in the ballpark. So that's huge. Um, but then that the downside of like I'm not that proficient in it. They're gonna then hire somebody profient in it or whatever and go down that path and's like ah I like I like the label world livewire gives me especially with livebar 3 especially so much you know. Cleans up a lot of the like rough edges and it's like all right? Well my one my one language should do everything will be php with like ah you know the smattering of Javascript where you have to get in there and do some javascript these stuff. So I know Kib's working on the component library for livewire which will be super awesome like cannot wait for that I think that'll be huge for.

37:25.75
Chris Morrell
Yes.

37:29.94
Ian Landsman
Ah, whole ecosystem. Honestly, Um, but I can't I can't wait for that either. So I won't get to use it probably or I'll Retro Fittt it in later. But but yeah, it's fun.

37:40.10
Chris Morrell
Sure? yeah, it's funny I still I still haven't used livewire I mean you know I've I've played with it. Um, and I was I was talking with ah I don't know I can't remember what we were talking about but Daniel and I were talking about livewire a couple months ago and he he was like. How is it I think I'm like the you know in the top twenty contributors to livewire and never actually used it because he and I like worked on ah a project when he was at Innachi where we were like experimenting with livewire so we like contributed a bunch of features and then never. Ah.

38:04.80
Ian Landsman
Built it.

38:11.10
Ian Landsman
You were in there fixing stuff. Yeah, yeah, well it happened a lot.

38:17.13
Chris Morrell
Never actually pushed him to production but it does I mean it does seem like ah, an amazing an amazing option it just it keeps on not quite fitting and I I don't know I I still love just lair Vllin Blade Honestly like just the beauty of.

38:31.34
Ian Landsman
Yeah I do love that too.

38:35.32
Chris Morrell
Just sending down a single request response like I don't know it's a lot of times. It's just so simple. Yeah.

38:37.70
Ian Landsman
Yeah, let's just so clean. Yeah, that's the thing It's like there's no, there's no place for anything to hide like if there's a problem like you know where to start looking right away whereas definitely all these other systems whether it's spa or it's a livewire or whatever there is these like.

38:48.80
Chris Morrell
Right.

38:57.49
Ian Landsman
Where is this problem like what is is it rendering wrong is it the data and I got to be in there looking at what's getting passed back and forth and try to figure out what's going on and like so right right? where right? What component is making this request like yup.

38:59.25
Chris Morrell
Yep, yeah.

39:04.90
Chris Morrell
Where is even this request getting made you know or or something that I was really I was kind of baffled by is like do you know where the livewire routes are being registered in your application because that's like.

39:22.96
Ian Landsman
Right? Yeah and that everything goes through 1 route like all livewire like secondary connections. Go to this single route which actually causes a little bit of problems here and there if you have things like looking at the current route and stuff. It's like well the current route is actually livewire update but like that's nothing. So.

39:23.40
Chris Morrell
Even that's not obvious you know it's like.

39:35.33
Chris Morrell
Right? right? right.

39:41.40
Ian Landsman
Um, there there are like weird edge cases like that for sure I think much like anything else like I've definitely found this in this new project here being all in on it actually cleans up some of the ugly stuff or like like every page is.

39:52.49
Chris Morrell
Oh I'm sure. Yeah.

39:58.76
Ian Landsman
Ah, full page livewire component and like it just simplifies a lot of stuff to some degree but still you still there are still some rough edges and still mostly the biggest thing I find with livewire is just like there are times where it's just hard to know. Yeah, what exactly is going back and forth and like.

39:58.83
Chris Morrell
Right.

40:16.87
Ian Landsman
Because of how it works with like bundling everything together and stuff. It's not that easy to even just like directly look at even if you're in like the network stuff so you know I think that's some that I know Caleb has some ideas around you know, having a more visual ah kind of representation of that and some like.

40:21.90
Chris Morrell
Right.

40:34.00
Chris Morrell
Right.

40:35.72
Ian Landsman
Dev tools and things which I think when it gets to that point will be super handy and really make it a lot easier for people but but you know it's just kind of like anything else I mean there's always there's always something you know it's always something. Ah.

40:49.36
Chris Morrell
Um, sure. Yeah no I mean I I totally get that though if it's if it's like the primary interface of ah you know this is I think this is this was kind of the thrust of the last episode we were talking About. Um. And we were talking about the scope of Blade components but the real conversation was just like just picking what you're going to do and doing it. You know it doesn't even matter what the what the choice is in the end. It's like just just making the choice and sticking with it because ah.

41:15.18
Ian Landsman
Right? Ah yeah.

41:24.22
Chris Morrell
Yeah, if you don't know okay is this a livewire thing or not That's so much worse than just always knowing that it's it's livewire. Yeah now.

41:32.42
Ian Landsman
Yeah, yeah, yeah, picking I mean that's something I've been better at times but um, but like picking something and just going is a huge huge benefit to what whatever you're talking about but getting started is is good.

41:40.77
Chris Morrell
Um, yeah, yeah, well and I think it's I think it's harder to do over a long span of time because the options change so dramatically and and the tastes and you know the ecosystem and and you know.

41:50.20
Ian Landsman
In.

41:59.85
Chris Morrell
Options. It's just like ah you know what we did even ten years ago is so different than you know what? the best practice is now and to some degree you have to keep up with that you know because other ease just start falling behind. So it's it's a balance.

42:14.34
Ian Landsman
Yeah, yeah, there's like yeah I was just to say like there is that balance of like you know I think there it's it's easy nowadays to go to.

42:18.72
Chris Morrell
Um.

42:28.30
Ian Landsman
Are and not like you're on Twitter you're on wherever and it's like everybody's doing all this hot stuff and it's amazing. It's like well I should be doing the hot stuff but then obviously like the actual economic value to your customers and yourself is you know how how much money is anybody ever made because of like a react.

42:30.10
Chris Morrell
Oh hundred percent

42:44.30
Ian Landsman
Ah, but even like framer or something like that where like oh everything's flying around the screen. It's all like it's bouncing and like whatever like but like that's literally nobody's ever made a purchasing decision at least about B Two B app based on like our things flying around the screen nicely like it's just never I mean every B Two B app you use. That's ah Enterprise app is like.

42:49.30
Chris Morrell
Yep.

43:03.40
Ian Landsman
Like truly an enterprise app is the most horrific thing you've ever seen like if the app costs more than $1000000 it's literally the worst app you've ever used and like that's what they're selling for millions dollars so you know there is it is easy to get caught up and be like oh my thing is bad because it's like it's not all flying around the screen but really.

43:04.85
Chris Morrell
Yes.

43:12.59
Chris Morrell
Yeah, yes.

43:22.29
Ian Landsman
Fine and nobody cares in the real world. But.

43:25.70
Chris Morrell
I mean I do think I do think that there's there's an element of I I think I will make decisions based on the the sort of perception of attention to detail that might come with um some of those less obvious things.

43:40.34
Ian Landsman
Yeah.

43:42.21
Chris Morrell
You know, a good example is ah ah, do you use linear linear. Um for okay yeah I mean okay, well ah I Definitely think that you know when we switched from Jira to linear like the the.

43:45.50
Ian Landsman
Sort sort of we're sort of playing around with it I'm not totally all in yet.

43:56.74
Ian Landsman
And.

44:00.81
Chris Morrell
Part of the reason that we made that switch was all of the little things that just were nicer. You know and and a lot of that really was speed. Um, but it also was just you know the Command K interactions were really.

44:07.86
Ian Landsman
Um, yeah.

44:18.46
Chris Morrell
Felt really good and looked really good and always had all the options that you wanted and just like navigating around felt really fluid and it was nice to look at like I think those things do matter but 100% at a certain at a certain scale. They stop mattering you know like.

44:33.24
Ian Landsman
Right? I think it's also ah, over time you know? yeah like it's like now like that's why our like the new version will be livewire right? because I think there is an expectation of like that there is a more instantaneousness to when you click a link or something like that right? So like there's that.

44:35.54
Chris Morrell
When when you're solving just like.

44:45.62
Chris Morrell
Yeah, yeah, yeah, yeah.

44:52.11
Ian Landsman
There's like that there's that level which I think has now become more table stakes and then there's like the next level which is like well every little thing something floats or like Bolt is beautifully right? like and I don't know if that's really selling a lot of software like um.

44:55.56
Chris Morrell
Yeah, yeah.

45:01.36
Chris Morrell
Yeah, yeah, sure. Yeah.

45:10.92
Ian Landsman
And and all my customers now don't have any of that right? and so you know there is like and linear like here like this is nothing linear has all that stuff down really well. But what they don't apparently have is real time database backups because like they crash and lost customer data.

45:12.12
Chris Morrell
Right.

45:27.59
Chris Morrell
Ah, yeah, yeah.

45:28.78
Ian Landsman
Because when they restored their backups aren't real time and so they're taking it from like a 12 hour old backup and I'm like well that's bizarre like I have real-time database backups like point in time recovery. Why don't you have point in time recovery database backups like that's weird. So you know I don't know like there are just things like that that make you say.

45:39.61
Chris Morrell
Alright.

45:48.75
Ian Landsman
A little bit odd like where people are prioriti I think things. Yeah, it's very scared I I make think super important in there right now and I'm like how does this like this is the hotness. This is like the big thing and they don't even have like.

45:50.19
Chris Morrell
Yep, now that was a little scary. Yeah.

46:03.33
Ian Landsman
Point and if you're and if you're not going to point in time like you have to go 12 hours back like that's a long time back. That's not like well it's an hour back like that's a whole day like if you were doing stuff that whole day. You just lost a whole day. All everything you logged in there. It's just gone like I know um, but anyway that's a little ah off topic. But.

46:06.34
Chris Morrell
Yeah, for sure.

46:15.76
Chris Morrell
Yeah.

46:21.66
Ian Landsman
Um, yeah.

46:22.68
Chris Morrell
Going back to caching ah something that I think I've noticed is not like ah maybe I'm just missing it. But I just don't feel like I hear people in the Lairvelle world talking about it at all is ah the htp side. You know, really managing the cache control headers and you know I would bet that there are very few web developers let alone laravel developers that understand you know what an e-tag is what's different like what is a week etag you know and.

46:39.56
Ian Landsman
Um, yeah.

46:55.10
Ian Landsman
Right? yeah.

46:58.15
Chris Morrell
It is ah cache control versus expires like you know all all these different all these different headers that you understand and and honestly you know I've been pretty deep in that stuff for a long time and I don't I don't ah fully understand it either. Ah nor do I really understand.

47:00.19
Ian Landsman
About.

47:11.61
Ian Landsman
Yeah, yeah.

47:16.11
Chris Morrell
The Ah, you know the shift from sort of like older headers to to the more ah modern headers. But that's something that for certain parts of our site I think we get a lot of value out of just. Thinking a little bit about how we can cash it. But it's the same. You know it's definitely the same tradeoff of ah then you get folks who like it's not changing and you have to explain. Okay, you have to hold shift and commit. Yeah, the it's definitely I mean caching always is a double edged sword and.

47:40.91
Ian Landsman
Right? Ah clear your browser. Yeah I.

47:48.31
Ian Landsman
Now.

47:50.76
Chris Morrell
I Don't even know I guess in the world of lair of a livewire That's just not a thing I guess on the initial render it still can be but for the rest of the request lifecycle.

47:58.11
Ian Landsman
Um, yeah, you could deal with the initial render could you do it I mean I suppose you could do it on a turn value I Wonder how that would work possibly.

48:09.14
Chris Morrell
Um I don't think so because like the ah you know the the yeah and the um, the ah, whatever it's called the hash of.

48:12.81
Ian Landsman
It might be sending live by itself is probably sending some type of no cash like request.

48:25.18
Ian Landsman
Checksum yeah, the checks some yeah and it might actually might always be posting to I don't actually recall um, ah yeah for our stuff I haven't thought about that as well I have thought about a lot on 1 hand but on the other hand for the app itself.

48:26.00
Chris Morrell
Yeah, the checks on stuff like that's going to right? like.

48:42.92
Ian Landsman
Ah, feel like it would be too far. Um I think that the the having people yeah like having like the your filters come back from like stay in the browser cache would probably be too far but um.

48:44.75
Chris Morrell
Um, yeah, that's more for like the front end marketing resources and stuff with that.

49:00.13
Ian Landsman
But yeah for so for helpspot.com the like marketing website is all fully cash like so this is was a beef I had with Aaron because he has this whole convoluted thing to make a ah ah ah, what's I'm looking for the right. The stack site generator. Yes.

49:11.30
Chris Morrell
His his not his his totally normal static site generator right.

49:18.95
Ian Landsman
So um, whereas like so I what we do helps all dot com is just a regular layerable app that just renders normally um and then behind Cloudflare I Just told cloudflare to like cache everything forever and then the only thing we have to do is when we um, you know, update it. We just.

49:26.53
Chris Morrell
Yep.

49:36.15
Ian Landsman
Clear the cloudflare cache and that's it So like so everybody gets not only a cash versions but like the cloudflare delivery network where it's coming from. You know their next door Neighbor's house or whatever. It's like super close and yeah and for stuff like that like it's great like everything is instantaneous. Um.

49:44.20
Chris Morrell
Sure.

49:54.32
Ian Landsman
It's just always cached and always on the edge. So yeah I think for that that kind of thing. It's super good and obviously like assets and stuff like that too just being careful that those are cached and things. Um, there are some like.

50:04.40
Chris Morrell
Sure now with that. Are you are you only setting the the cache control headers for the that the the shared headers and leaving the um. Yeah, letting the browser always load from cloudflare. Are you also letting the browser cache um those pages I'm sure. Yeah.

50:24.90
Ian Landsman
Um I think it is a while ago now and I honestly couldn't tell you I think the browser is caching I Want to say but I could be wrong which I guess that could be then a problem. Ah there a little bit but I don't cheese now don't know.

50:39.44
Chris Morrell
Yeah I mean typically you'll have you know there's like the the shared cache and then there's the private cache and and the shared cache is is telling the intermediate proxies what they can keep and then the private cache is just the end user. What their browser can keep.

50:41.46
Ian Landsman
Can tell you years ago now right.

50:59.60
Chris Morrell
And yeah, you know we'll we'll have things where it's like you know at the browser level cache this for a minute and then at the um intermediate level cache it for 5 minutes or something like that. So like um and and but it is it definitely comes with tradeoffs in terms of.

50:59.21
Ian Landsman
Yep.

51:06.71
Ian Landsman
Right.

51:17.53
Chris Morrell
Ah, freshness you know it's always. It's always tricky you. You have to make that request somewhere. Yeah.

51:21.78
Ian Landsman
Yeah I would be worried the support the support end of that for the like for the marketing site. It's like but we're not even updating it that much So it's whatever, but um, but yeah for the app itself. Ah, you're pretty brave out there doing the.

51:26.57
Chris Morrell
Right.

51:36.82
Ian Landsman
Doing the browser-based caching inside the app I like that.

51:38.50
Chris Morrell
Well no I mean so we actually drive the marketing site and the app all from the same from the same codebase so most of the stuff that we do with cash control. Um is you know our frontend pages that aren't changing very often.

51:45.52
Ian Landsman
Are.

51:53.13
Ian Landsman
Yeah, yeah.

51:56.29
Chris Morrell
Um, we might you know like there's ah you know the the user's like main dashboard view that just is kind of like here's all the stuff that you have access to and this is like what course you're in the middle of to pick to continue from and the handful of things like that that that content doesn't change.

52:06.84
Ian Landsman
What's going on or whatever. Yeah.

52:15.62
Chris Morrell
Ah, very frequently. You know it changes but it's only going to change every ten ten minutes to an hour for any given user that we do cache because um, we do find that folks you know will navigate back to that 1 page pretty frequently. So just yeah, it's I mean it and it.

52:20.53
Ian Landsman
Right.

52:28.10
Ian Landsman
So it can just be instant. Yeah.

52:33.44
Chris Morrell
Upside is it is literally instant because it's just loading from file a local file system. Um, but it feels like a full page load. Yeah I think that there are a few places. Um, where it pays off depending on on your access patterns. But it's it's definitely.

52:35.76
Ian Landsman
Right? from level. Ah I'm intrigued by this idea.

52:52.30
Chris Morrell
It's even scarier than um, you know serverside caching because once you've made the decision. You can't back out of it. You know unless you're like deciding to change the yeah url of pages or something like that and you're you're totally screwed you know? Um, but.

52:56.59
Ian Landsman
Yeah, it's out of your hands right? Yeah or you're you're panically changing the url of a page to break. People's caches where you set it for a year instead of a minute.

53:10.58
Chris Morrell
Yeah, yeah, yeah, that would be funny to just prefix every single url with some like version prefix and then if you ever need to? yeah.

53:16.52
Ian Landsman
Um.

53:21.29
Ian Landsman
So you can never go too long. Yeah, that's kind of interesting. This is like more.

53:28.10
Chris Morrell
Slash V Three slash dashboard or whatever you know? yeah.

53:33.62
Ian Landsman
Big brain stuff that I can't really get into but it does seem like there are like always elements of pages I would love to cache. Um, that's even more like even in blaed like I wish there was a cache bleed component or something like a ah directive um because like yeah this chunk I know this chunk is like yeah some like.

53:39.85
Chris Morrell
Um.

53:52.29
Ian Landsman
Ah, whatever Sidebar or something that I know I don't need to update too often and rather than going down into like every query that makes up the sidebar and like dealing with all that caching like no when we render this. Let's just like cache this chunk. Um, there is like I think there is a lot of package out there for it. But I don't know it seems a little.

54:04.77
Chris Morrell
Yeah.

54:11.27
Ian Landsman
Don't like they' like have too many kind of random packages and didn't seem like you know well I wasn't sure what's going on with it and I was like ah whatever I'm not going deal with it but stuff like that is kind of intriguing to me of like can we just cache the html output of this chunk and you know not have to rewrite because some of that stuff's heavy like or even like your top navigation bar.

54:12.50
Chris Morrell
Sure sure.

54:22.88
Chris Morrell
Right.

54:30.42
Ian Landsman
Like there's actually like a fair amount of stuff in there often. That's not updated that much. But it's like this big chunk of html that's generated from a bunch of loops of various things and whatever and it's like well maybe we could cash that whole chunk.

54:31.22
Chris Morrell
Yeah, for sure.

54:41.53
Chris Morrell
Yeah I would say our our our dashboard our navigation. Um for the logged in app is probably the place that we we cache the most so you know anything? yeah anything that's going to show counts or anything that's going to be.

54:51.73
Ian Landsman
And.

54:59.53
Chris Morrell
Um, you know, based on access levels. Um, pretty much all of that We cache those queries. Ah and then just store like ah ah a single cache payload of like a bunch of numbers and booleans that we. Then use in the logic of rendering the sidebar because yeah, that's something that is surprisingly dynamic and is rendered on literally every page. So Yeah and that one.

55:18.63
Ian Landsman
Yeah.

55:26.18
Ian Landsman
Yeah, yeah, lot of bang for the buck there on those cash in those items? Yep yeah, that's the thing with like help this because like there's lots of those lots of counts on the sidebar.

55:36.74
Chris Morrell
Yeah, yeah, yeah, yeah, and that that one is I think um, it's pretty easy to do that like sort of grouping technique where we basically just have a couple of buckets. There's just like a sidebar. There's like a bucket that's.

55:46.16
Ian Landsman
Um, right.

55:55.23
Chris Morrell
Given your role within the application. Everyone with this role sees these same things so we can just like kind of cache it for that entire role and then give and then individual users will have their own cache for their own individual views. Um, and that way.

55:58.28
Ian Landsman
Right? Boom yeah.

56:15.22
Chris Morrell
When something changes we just you know destroy the whole the whole cache for that role or and then when when anything related to those those counts or those those flags changes. We just you know in a in a observer we'll just clear that cache out.

56:20.87
Ian Landsman
Yep.

56:31.21
Ian Landsman
Where do you guys cash with like Redis or yeah standard stuff.

56:33.50
Chris Morrell
Um, yeah, yeah, Redis ah elastic cache on Aws yeah I've thought about I mean because is that what you're using as well elastic cache or or ah managed Redis or one of those.

56:47.36
Ian Landsman
Ah, right? Well I mean on the existing help spot. It's actually I'm using the oh no now it is on Redis I think for a long time is on the file because each is file just file caching actually and each customer has their own server. But now this each customer server has redis.

56:57.32
Chris Morrell
Right? sure.

57:04.54
Ian Landsman
Um, and I believe the cache is in Redis there actually now. But that's like what I have penciled in no final determination on kind of the next generation. Um, we're probably going to be using single store which could in theory possibly even like use the in-memory.

57:20.60
Chris Morrell
Right.

57:21.52
Ian Landsman
Tables with like a key value roll our own sort of thing but on the flip side I feel like half the reason I want the cache is just to like avoid hitting the database even if it's just for stupid basque queries. So ah, yeah, most likely will be ah will be redis.

57:27.90
Chris Morrell
Right? right.

57:36.82
Chris Morrell
Something I've thought about is like and they talk about really overengineering what what what about um running excuse me a local copy of Redis on each application server that you have. That use that you cache and then either I don't know which would be more performant either sort of hopping through 2 layers of cash right? where you have your local app cache where it's just like you. You'd only you could just have a you know. Hundred megabytes like you wouldn't have to have much ah much in there but just like or probably even less than that. But just ah, you know, just a little space for the really really frequently access stuff and either just make that just decide. Ah, which.

58:14.35
Ian Landsman
Right.

58:24.30
Ian Landsman
Um, yeah.

58:32.78
Chris Morrell
Um, cash location. You're reading and writing to or potentially yeah or or potentially like what would it be look like if you just had introduced an additional layer where it's like ask local first is it' still going to be faster to ask the shared cashche um later and.

58:33.38
Ian Landsman
Right? If you want to go local or.

58:44.61
Ian Landsman
Yeah, for sure.

58:52.10
Chris Morrell
Then if the shared cash doesn't have it do the query. Um, and you could have like you know, maybe the shared cash holds it for 5 minutes and the local cash holds it for a minute. Um, and you're you know there's there's something interesting there but I don't know if the overhead would be worth it.

58:53.30
Ian Landsman
Yeah.

59:09.43
Ian Landsman
There is I mean if you made the session sticky to the app server then like it would be faster access wise and they'd always be getting the same server. Obviously then like if there's a problem with that server then that's a problem. Um.

59:11.61
Chris Morrell
I haven't explored it at all. But.

59:21.27
Chris Morrell
Right? right.

59:27.46
Ian Landsman
But even if you don't have many frontend like if you have 3 frontend servers you're still be reducing your queries a fair amount even if their're own even if sometimes it's duplicated into cache or whatever across different requests. Um, but it does feel like maybe maybe it's a bridge too far. It does feel like it might be overengineered.

59:38.66
Chris Morrell
Um, yeah, yeah I Just don't know if the yeah right.

59:45.49
Ian Landsman
Potentially I do love the idea of local caching I hate going out over the network like I just want it to be all all right there I mean the other annoying thing is I really feel like things are so fast right now it's like I do have this temptation which I'm not going to do cause I'm a chicken but I do. Like you could just run the whole thing on a single server like you could run the whole thing on 1 badass single server for like quite a while. Um, especially if the database wasn't on there then you really could ah but I don't know.

01:00:02.74
Chris Morrell
Yes.

01:00:14.95
Chris Morrell
I mean.

01:00:16.53
Ian Landsman
Obviously then then server has a problem and you're annoyed but then servers are pretty reliable now. So you're going to go a while probably without a problem. But.

01:00:24.13
Chris Morrell
We we just decommissioned a server that was running ah red hat four. Ah, it's ah from to 2012 I think and it's just been just been running fine doing its thing no problem.

01:00:34.71
Ian Landsman
Right? Ah, just been doing its thing. Yeah.

01:00:41.15
Chris Morrell
Never gave me any issues you know, just never had to think about it and ah yeah, we're we're pretty much all in a a Aws now. Um, we've hopped around over the years um I mean I don't think I don't even think.

01:00:43.30
Ian Landsman
Is all your stuff on Aws or do you have like physical servers or what do you? what you doing there? yeah.

01:00:59.10
Chris Morrell
You know Aws was an option when we started. You know it was like a ah New Jersey colo of some sort that we were on like back then and.

01:00:59.45
Ian Landsman
Right.

01:01:07.22
Ian Landsman
That's what I was like just for the marketing site I bought a server and sent it to a colo because I was like there was no really anywhere you can like even have a marketing site like you had to have a server so that's what I did.

01:01:13.98
Chris Morrell
Right? right? Yeah yeah, and then ah we moved we moved to Rackspace for a while. But um I I found the fanatical support not as as incredible as ah.

01:01:30.41
Ian Landsman
Not fanatical. Yeah.

01:01:33.13
Chris Morrell
They really wanted to to sell you on and and the the price of that fanatical support was not insignificant. So um, yeah, we switched to Aws probably a little bit before we switched we started switching over to la l so it's been. We've been on ado for a long time now and I love it I mean.

01:01:39.34
Ian Landsman
Yeah.

01:01:48.81
Ian Landsman
Yeah.

01:01:52.60
Chris Morrell
You know it's just like sure we spend a lot on our a to Bs bill but it's just like all of it is just managed you know rds and elastic cache and E C two instances and like we've invested a lot of time in. Um.

01:02:03.74
Ian Landsman
Well.

01:02:11.30
Chris Morrell
We we don't have ah like infrastructure as code like a terraform type situation. But we have a really solid ansible setup. So like when we need to deploy a new like a new version of Pgp. For example, we'll just provision a couple of servers. Ah you know.

01:02:13.76
Ian Landsman
Right.

01:02:28.75
Chris Morrell
And and run all those ansible playbooks on those new servers and then just flip ah flip the switch and the load balancer and it's just like easy. You know Aws just makes everything easy. Um.

01:02:32.10
Ian Landsman
Um, yeah.

01:02:36.31
Ian Landsman
Yeah, yeah, no, we've been very happy that Ws I don't have any complaints. It is a little pricey on some points but it's like what are you gonna do. It's fine. And yeah, we do the same thing with ansible for a lot of stuff and we are moving.

01:02:54.60
Chris Morrell
Ansible is great I don't feel like I hear people talking about it very often. But I love I mean we I have a ah little script you should check it out. It's called ansible interactive but it just ah, basically like gives you a nice little.

01:02:55.36
Ian Landsman
Yeah, it's been pretty good.

01:03:05.48
Ian Landsman
E.

01:03:11.53
Chris Morrell
Command line ah selector tool to pick like you know which which groups do you want to deploy which which tags which plays um but I mean we've been deploying with ansible for years now and it's just like it's just such a.

01:03:18.23
Ian Landsman
Go.

01:03:28.54
Chris Morrell
I mean just compared to the old days when it's like how did I configure this server like what do I need to remember to do next time or you just like have a note somewhere. That's like remember you have to install this this weird like you have to pin this one dependency because.

01:03:32.23
Ian Landsman
Just got to remember all this stuff. Yeah.

01:03:40.57
Ian Landsman
O.

01:03:45.25
Chris Morrell
For some reason like in this distribution you can't use the latest one you have to like use 2 points before you know all that stuff. It's just like all that is just committed to get and it always just plays out the same way I I love it.

01:03:47.14
Ian Landsman
Now now.

01:03:59.19
Ian Landsman
Yeah, yeah, yeah, same and yeah, that's what we do for like these 3 400 servers whatever it is. It's like just ansible spins them up and all that kind of stuff. So yeah, there's ah it's really great.

01:04:10.88
Chris Morrell
Um, what's that look like do you is that manual or do you have a whole ah P B driven ah so is that it's ph p drives ansible which provisions the servers. Yeah.

01:04:16.44
Ian Landsman
No, it's like a hole. Yeah, there's like a the currently yes, we are doing some work to like move that um to like Aws is terraform or one of the I'm not the one doing it.

01:04:29.69
Chris Morrell
Okay, Cloud Cloud formation I think it's called yeah.

01:04:34.34
Ian Landsman
Whatever system cloud formation. Yeah maybes confirmation. Um, but 1 of the guys who works at you Justcape Matt is working on that as a project um to kind of get that more because it's a little bit like it's just it could be more automated if it's inside Aws um, so.

01:04:44.13
Chris Morrell
Ah.

01:04:49.28
Chris Morrell
Sure.

01:04:54.10
Ian Landsman
Just getting that all set up because eventually like when you get the next generation help spot out which is going to be a little while from now. Ah, but you know we want to make like the managing of the existing platform a little just more seamless and because you know over years. Basically we're going to do is like. You know, kind of like what basecamp did actually which is like if you want to stay on the old 1 You can just stay there forever and we'll just like maintain it. So um, so we're still got to manage that platform even when the new versions out. We're not going to just move everybody over. so um so yeah so just spending some time cleaning that up and modernizing it a bit.

01:05:16.50
Chris Morrell
Um, yeah.

01:05:30.52
Chris Morrell
Yeah.

01:05:32.60
Ian Landsman
but but yeah it's even our current system I mean it's been I guess close to 10 years or around 10 years um that's been operating and doing great. So Chris fidow built that originally way back in the day. Yeah, and now. Ah.

01:05:45.40
Chris Morrell
Okay, ah.

01:05:50.33
Ian Landsman
Matt Stenson it works me kind of manages it and does kind of work on the new next generation of that.

01:05:56.65
Chris Morrell
so so I didn't realize realize that Chris was also ah a user escape fellow. Yeah, there's a lot of okay there you go.

01:06:02.41
Ian Landsman
Yeah, he worked you escape for Eight or nine years maybe something like that seven or eight years um yeah so yeah, a lot of the larabelle old school labe came came through usercape. Yep yep, it has been fun.

01:06:14.78
Chris Morrell
Um, the O G folks. Yeah there you go. That's nice, Nice to nice to have. Yeah so what did um what did switching a airbell look like for you when you did it.

01:06:20.25
Ian Landsman
Kind of be in the mix. Yeah, it's and fun.

01:06:28.40
Chris Morrell
Like did you just adopt it into the existing application. You just kind of like started to bring in the laravell components.

01:06:35.70
Ian Landsman
Um I think very originally there was some of that and then we did a big project to like basically have Labelle be the foundation but then kind of take all the old stuff and put it inside Labelle which was just easier to kind of manage because we didn't even have like composer like it's all pre.

01:06:42.80
Chris Morrell
So.

01:06:52.42
Chris Morrell
Um, yeah, yeah, yeah.

01:06:52.86
Ian Landsman
All that stuff right? as you know? So um, this way we could like have composer and all that and ah, but then just have like the function libraries and things from the Og version still be used without having to like rewrite every single thing.

01:07:09.70
Chris Morrell
Right.

01:07:11.47
Ian Landsman
Um, so that's kind of where it is now it's like labels. Ah you know use like the core of labele. It does go through Laol routing but doesn't use blaed. Um, so things like that where we have some magic and tricks in there a little bit. Ah but you know we use like.

01:07:25.18
Chris Morrell
Yes.

01:07:29.66
Ian Landsman
Liables caching and all those sorts of things um come from Lael itself. But yeah, so it's like you would recognize it as a layerable app on the surface. But then when you dig in a little bit. There's some special sauce in there.

01:07:44.98
Chris Morrell
Does um like was the was the app before you made that switch like did you have any tests or any like reproducibility at the time. Well I guess because you were ah ah um.

01:08:00.34
Chris Morrell
What's the word that I'm looking for you were you were giving the app to your customers on Preise there you go ah you you kind of had to to be able to reproduce the distribution of that pretty easily did where you did you have a good test suite at the time did that make it easier to switch.

01:08:02.77
Ian Landsman
Oh on Premise yeah.

01:08:18.41
Ian Landsman
Oh no, initially way back in the day there was not much of a test suite at all. No for sure like um because that wasn't even a thing way back in the day really? Ah, but no since it's on Larryba Now there's it still doesn't have as much test coverage as I would probably like Definitely the new version have way more because they'll just.

01:08:19.75
Chris Morrell
Yeah, yeah, yeah.

01:08:31.62
Chris Morrell
So yeah.

01:08:35.89
Ian Landsman
Doing it from the beginning and all that but but there is way more now than than there was before for sure. Um, and so yeah I mean that that part good I mean the build system for like On-premise customers and everything has been was been automated for a long time in terms of like building.

01:08:52.87
Chris Morrell
Yeah.

01:08:55.30
Ian Landsman
Ah, new releases and packaging them up into different deployments. There's like for a long time. We had a windows installer because a lot of windows customers don't really want to deal with like installing php themselves and dealing with is and all this stuff So like it would.

01:09:02.50
Chris Morrell
Okay, yeah, sure do the whole thing.

01:09:11.28
Ian Landsman
You know install Hp into ios and install a database or connect it to sql server. Um, or Microsoft Sql server and whatever just do all this windows configuration now it kind of moved away from that because basically the people who are hosting on-premise now are they're pretty.

01:09:28.19
Chris Morrell
Yeah, yeah for sure.

01:09:28.43
Ian Landsman
re pretty knowledgeable like if you're hosting On-premise you're like a real it person. You know what's up like you want to man you want to be able to update your php yourself and things like that. So now we've moved away from the installer and just like have instructions on how to you know, set up the environment things like that.

01:09:37.34
Chris Morrell
Yeah, yeah.

01:09:44.31
Chris Morrell
Yes.

01:09:46.20
Ian Landsman
But for a long time. It was more of a middle ground where like people were buying it and just like dumping an I or and or like it was like some half baked it tea it wasn't like an actual real internal it and so those folks didn't necessarily want to. Dig in too deep and also like ph p in windows wasn't even like a thing. It was very much not a thing back in the day so it's more of a thing now with like the with Microsoft um, kind of embracing you know when they had ah.

01:10:03.79
Chris Morrell
Right? yeah.

01:10:15.14
Ian Landsman
Kind of change over to being like no instead of like being all on our platform. We're just going to be everywhere and we're going to support everything and part of that was like more enhancements for ph p and things like that. so so yeah so now the windows it people are kind of they're aware of what Php is at least and. Might have some experience with it and if they want to host on-premise they can set it up themselves and then they can manage when there's a dependency that has a security issue and you want to update or whatever.

01:10:36.26
Chris Morrell
Yeah.

01:10:43.19
Chris Morrell
Yeah, that was I mean I was just curious because that definitely has been one of the biggest pain points for us is yeah I mean we didn't have any tests for anything right? And ah you know like I.

01:10:53.62
Ian Landsman
Right? yeah.

01:10:59.90
Chris Morrell
I you know I joke like are I would say millions of page views a month were still running through just like a single like pages dot p h p p p four file ah not more not like 2 two and a half years ago it was still just like.

01:11:13.47
Ian Landsman
Might ah.

01:11:17.10
Ian Landsman
Right.

01:11:19.65
Chris Morrell
Happily chugging along but like the other problem is all of that code ah was basically all written with like specific path assumptions or specific configuration assumptions that. I Have no idea anymore. What any of those were so like we're in a position where our legacy code base is just like it has to be on this server and everything has to be in in the exact location that it is right now because otherwise everything will break. You know it's like ah.

01:11:37.85
Ian Landsman
Right.

01:11:49.74
Ian Landsman
Um, don't touch it. Ah well I mean you've been running I mean a Sas like service for like a very insanely long time like that's pre that term of Sas I would say right because it was never on-premise right? It was always hosted. Ah.

01:11:58.38
Chris Morrell
Um, yeah, yeah, no, no I mean yeah, we're not you know we operate like a saas but we aren't um because we're a membership organization I don't know. Ah.

01:12:08.71
Ian Landsman
Dry it.

01:12:15.23
Chris Morrell
Ah, we're ah we're a trade association for home inspectors. Um, and so you know we started mostly on. Um, you know membership benefits and like the application process that was like really what our first? ah Php application was.

01:12:19.70
Ian Landsman
Ah.

01:12:28.83
Ian Landsman
Right.

01:12:34.62
Chris Morrell
And you know I think if if you think about like the the late 90 s early two thousand s it was just like it was just that moment when people were starting to get comfortable doing a little more than just like browsing the web right? There was still a lot of you know, like a lot of hesitation about.

01:12:45.76
Ian Landsman
May.

01:12:51.69
Ian Landsman
A.

01:12:51.87
Chris Morrell
Paying for things on the internet right? but that was starting to go away. You had to have those big like you know Verisign seals and you know all that stuff mattered a lot. But um, yeah, like we we kind of rolled our own and perhaps you know.

01:13:00.13
Ian Landsman
Oh yeah, um, yep.

01:13:11.12
Chris Morrell
Certainly not the first but like 1 of the first e-learning platforms that that existed on the web like that just wasn't a thing yet, you know, um, and so yeah, we like we kind of operate as though we're a Sas. Um, but um, you know we're a nonprofit organization. Yeah, um.

01:13:16.41
Ian Landsman
You know.

01:13:24.14
Ian Landsman
The business models not assess. Yeah yeah, actually that's where I learned the program on the internet was elearning platforms in the late you know, not nineteen Ninety Nine Two Thousand whatever it was 2001 nope

01:13:30.60
Chris Morrell
And so oh really? yeah yeah I mean I remember like just God wait We could just build what we can just build whatever php just lets us.

01:13:48.48
Ian Landsman
Ah, yeah, that's all you need ah crazy those were wild times. Ah.

01:13:49.34
Chris Morrell
You know you got beach p got mysql like I can figure it out sure like we could bake whatever happened. Ah it is wild. Ah, but yeah, the the switch. So I think because of that the switch for us. Just felt like this idea of trying to move the existing codebase into a laravell app or um, bring laralle into the existing code basee I mean we do. We do actually have like a handful of Laravelle packages. It's. You you know they're like Larel 5 because that servers still running Pp 7 um or is it I mean I remember upgrading to 5.6 you know was like kind of a miracle and then I don't i.

01:14:30.10
Ian Landsman
And.

01:14:45.26
Chris Morrell
Nobody listen but that might still be running five six I'm not sure. Um, but ah, ah, we're you know so we're running some LaraL five stuff on that server. We've kind of like managed to fake. Ah you know fake the user model and ah get a little bit of like eloquent.

01:14:47.47
Ian Landsman
Right? ah.

01:14:55.93
Ian Landsman
Um.

01:15:01.50
Ian Landsman
Try it.

01:15:04.82
Chris Morrell
And a little bit of a blade in there which was kind of an interesting experience I Really do appreciate all the work that has gone into like making all those packages independent because you really can just pull in Blade and make it work. You know.

01:15:05.77
Ian Landsman
This.

01:15:18.84
Ian Landsman
All right? It'll do stuff.

01:15:22.55
Chris Morrell
Yeah, it will do stuff it will render. It'll render a page for you and ah, it's pretty cool to have um, you know we the the first piece that we need to implement was basically writing a ah compatible authentication system.

01:15:26.69
Ian Landsman
Ah, that's going crazy.

01:15:39.10
Ian Landsman
And.

01:15:41.26
Chris Morrell
Right? So essentially we needed authcookies from the legacy system to let you log into the modern system and auth cookies from the modern system. Let you like log into the legacy system. Um, but once that was in place you know now in that legacy code you can actually like. Call Auth Colon and cold user and you get back a user model that looks or a user object that looks like a user model. Um, we have a little bit of that. But it's it's minimal and it just felt like it was an easier move to just kind of do that clean, split.

01:16:06.90
Ian Landsman
Right.

01:16:17.70
Ian Landsman
Now.

01:16:19.72
Chris Morrell
Um, and it also allowed for a lot of like everything is tested in the new application. You know like we could kind of just adopt all the best practices ah from Clean slate almost like a greenfield application. Um, but just do it piece by piece and we have to deal with a lot of like ugly.

01:16:28.21
Ian Landsman
Right? by? Yeah, ah yeah.

01:16:38.31
Chris Morrell
Models you know models that are not ah necessarily you You wouldn't design them that way now. But ah, you know there's still eloquent models that look and feel like it. Um, that makes me wonder you talk, you've talked a lot about your um.

01:16:43.88
Ian Landsman
For Sure. Oh.

01:16:56.67
Chris Morrell
Sort of like old school Sql mindset. Um, how much do you find yourself like wanting to hack ah models and model relationships. Do you ever like have you ever written like a custom relationship for larevell or um.

01:16:59.40
Ian Landsman
Um.

01:17:14.10
Ian Landsman
Um I don't think so not like anything crazy custom like maybe add some ah some constraints on a model relationship or something like that but nothing too like if I.

01:17:16.30
Chris Morrell
Now.

01:17:25.90
Ian Landsman
Um, in like that zone then I just all go down to like the query builder directly and just like do whatever I have to do for the most part I don't think I've ever gone too far into like custom relationships really.

01:17:34.00
Chris Morrell
Ah, it baffles me I like that's something that I want to reach for all the time and I've never I've never met anyone else who ever has done it before like I'm just like yeah.

01:17:42.93
Ian Landsman
It really get so caught up with like we're hitting the Ma We can't leave I look like whatever just run the query like.

01:17:50.66
Chris Morrell
I own. Well no I will just I'll I'll take ah I'll take the time to like build a full custom like ah you know I might I might like extend as many or belongs to or whatever but I'll actually. Implement a full custom relationship that handles all the joins all the like you know eager versus non eager versus count like all that context and I I find that it's you know it takes it takes an extra effort to do that work at first. But.

01:18:14.13
Ian Landsman
Are ah yeah.

01:18:25.49
Ian Landsman
Yep.

01:18:26.53
Chris Morrell
You know, like an an example is we have you know we deal with a ton of state licensing so like ah states might have multiple licensing departments that issue licenses for courses for.

01:18:40.51
Ian Landsman
Who.

01:18:42.59
Chris Morrell
Um, inspectors right? or issue licenses and have continued education requirements and the department might issue a license and then issue ah education providers like a providership or something like that and um, so a given course ah through these approvals may have ah.

01:18:51.18
Ian Landsman
N.

01:19:02.36
Chris Morrell
Fifteen thirty approved departments right? departments that approve that course? Um, but it's not just like ah you know has many through because there are some departments that are blanket approval. Um, where.

01:19:05.72
Ian Landsman
Um, earth.

01:19:18.31
Ian Landsman
Ah.

01:19:19.51
Chris Morrell
Every course like we have a good relationship with the state. Every course that we release is just approved by default. Um, and there are other approvals that like have some some additional constraints and so you know we have ah this class this just has approved departments.

01:19:26.19
Ian Landsman
Um, yeah.

01:19:32.41
Ian Landsman
Yep.

01:19:38.96
Chris Morrell
And it does this bonkers. You know, join Union like ah computed column nonsense. Ah, but the result is you can just call approved departments on a course.

01:19:42.57
Ian Landsman
Ah.

01:19:47.79
Ian Landsman
I.

01:19:54.54
Ian Landsman
Then get him right.

01:19:55.88
Chris Morrell
And you just get all the departments that approve that course and you can like load 15 courses with approved departments and you get a nice ah, really efficient eager load out of the box and I I find that a lot of times taking the time to. Put something like that together and like staying in eloquent is really nice I will I will say but um.

01:20:13.89
Ian Landsman
You know I'm gonna have to give that some thought here when I hit these things eventually in the next Gen here because like I guess my initial reaction to that type of thing tends to be like. To build an action or something that just handles this scenario and does then whatever it needs to do in its own zone and then like I'm just going to call that action and get whatever I'm trying to get back but not to extend eloquent in that way would not be my first thought for sure. Um, that's interesting.

01:20:36.68
Chris Morrell
Oh.

01:20:46.73
Chris Morrell
Yeah, that that's one of my oh go ahead? No I just that and circular relationships. These are 2 things that I find myself just bumping into nonstop and it just.

01:20:48.43
Ian Landsman
I Do that's interesting. Do you but no go ahead. You guys.

01:20:58.49
Ian Landsman
But.

01:21:04.78
Chris Morrell
Like I was talking to Aaron the other day I was like we should do an episode on circular relationships and he was like I've never I've never had to deal with that in my life. It's like how how how is that true like okay, ah ah, ah, user.

01:21:10.50
Ian Landsman
Right? I don't know if I have either to say I don't know I must have at some point maybe but.

01:21:24.77
Ian Landsman
Yeah, yeah.

01:21:24.80
Chris Morrell
Ah, is part of a company right? The company has an owner right? Um, if the if if the user is the owner I don't want to have to do like an additional query right to get.

01:21:35.68
Ian Landsman
Go move.

01:21:41.79
Chris Morrell
That original user model again or even worst case like you know, imagine that you have like a user and there's like a company memberships ah pivot table to companies and then the company you need the owner of the company that's through a company membership pivot.

01:21:52.30
Ian Landsman
Who.

01:21:58.95
Ian Landsman
And.

01:22:00.48
Chris Morrell
Back to the owner like the the relationship between the pivot has like their relationship to the company whether it's like a owner or manager or or member. Um, you know the way to to kind of do that efficiency efficiently with ear loads. You're essentially doing like.

01:22:07.15
Ian Landsman
Right in.

01:22:20.16
Chris Morrell
Like user with company membership dot company dot company membership dot owner. Ah and you and for a user that's a single member owner right? You're essentially loading the same user twice The same company twice in the same company membership Pivot table twice.

01:22:24.19
Ian Landsman
No.

01:22:30.80
Ian Landsman
I.

01:22:36.77
Ian Landsman
Right off.

01:22:38.97
Chris Morrell
Ah, and I don't know that drives me nuts. But there's really not Ah, there's not a clean way to do it because if you try to automate. Um, you know if you try to automate it. You start to run into like circular.

01:22:41.65
Ian Landsman
That would drive me. Not yeah I don't.

01:22:57.94
Chris Morrell
Reference issues where if you call to array and like the user references the company and the company references a user. It's just like an infinite loop. Ah and I just it's just one of those things that I'm like how is no one. How is no one else having this problem.

01:22:59.68
Ian Landsman
Um.

01:23:05.99
Ian Landsman
It gets it gets wacky. Ah.

01:23:11.96
Ian Landsman
I don't know it's weird must to be doing making exceptions in our database in different ways. So we don't have to deal with it somehow or there we're like now we're just not gonna deal with that because we're gonna put this data over here instead. So it's all in 1 spot or whatever or.

01:23:14.66
Chris Morrell
I Think about it all the time.

01:23:20.65
Chris Morrell
Maybe yeah.

01:23:30.12
Ian Landsman
Which can deal with the second query which I try not to do. But yeah.

01:23:31.34
Chris Morrell
I mean we do sometimes I what made me think of it is. There's just like I was looking at code and it just said there's like a comment above it. That's just this query is bonkers but it's bonkers for a reason and it was literally that what I just described like. Users that company memberships that company dot company memberships dot company dot owner or something like that you know like and that was the most efficient way to do things and it was fine. You know you're loading a couple things twice but in the end you know it. It ends up being like 4 queries and you're loading.

01:23:47.75
Ian Landsman
Ah.

01:23:50.82
Ian Landsman
Oh sure sir.

01:24:07.30
Chris Morrell
3 things each time. So it's not that big of a deal to load the data twice? Yeah yeah.

01:24:08.73
Ian Landsman
Yeah, it depends where it is. It's just for 1 user. You know it's like it's it's not something that's getting hammered. It's like is it worth the time to like for complexity to make it more efficient or restructure the database or whatever. It's like probably not like. These are all real-w world apps is what I like about talking to people who have real- world apps because it's like no this is like the ugly thing that somebody did twenty years ago and like we're just it's okay where it works. it's it's fine right but it still works.

01:24:35.30
Chris Morrell
Right? Yeah either either we had a good reason to do it that way back then or we didn't have a good reason to do it that way back then but we did it that way. Anyway.

01:24:46.53
Ian Landsman
Ah, it works. It's fine.

01:24:48.10
Chris Morrell
And now now like what are you going to do. There's a 100000 rows in this table and and 5000000 rows in that table and like I'm not going to like change it just for the sake of ah saying that it's a little cleaner. You know.

01:24:57.43
Ian Landsman
Yeah, yeah, and your data but data size is so important these things because it's like even that's not that much data you know and it's like you can get away with a lot right? like with 5000000 rows like if you have 50000000 rows it's like way harder and if you have 500000000 rows it's

01:25:06.68
Chris Morrell
Um, oh yeah.

01:25:15.24
Ian Landsman
Super hard Now you can't get away with anything right? So it's like there's all these different scale issues of like what what is involved at the different size databases and how much you can depend on like not even caring what eloquence doing because who cares it like returns me back to models I want and.

01:25:19.16
Chris Morrell
Absolutely.

01:25:33.25
Ian Landsman
Doesn't matter how fast it is because it's going to be passed no matter what verse. Ah yeah, a bad query obviously on 500000000 rows and it's going to be minutes of grinding away.

01:25:43.83
Chris Morrell
Yeah I mean people definitely overengineer that stuff all the time because like you know I think we're a pretty successful company. We've been around for a long time. Ah, but our users table only has a few hundred thousand records in it. You know it's not like ah.

01:25:48.66
Ian Landsman
Well.

01:25:57.73
Ian Landsman

01:26:01.62
Chris Morrell
Even after all these years. It's just not. That's not that big of a deal. You know like the the biggest tables that we're dealing with are like audit logs. Maybe um, which get which get pretty big. But um, you know, even the even the tables that.

01:26:05.64
Ian Landsman
Yeah. Yeah.

01:26:18.62
Chris Morrell
Yeah, do have millions of records in I'm like that's that's over dozens of years and like a pretty active pretty successful. You know product. It's usually you're not dealing. Yeah right right.

01:26:23.58
Ian Landsman
My.

01:26:30.34
Ian Landsman
Yeah, and on a modern database server like that's going to be pretty fast. It's probably even all in memory like it's going to be pretty fast. Um, yeah, yeah, yeah.

01:26:38.55
Chris Morrell
Usually you don't have to think about it every once in a while we'll bump into an issue and it's like oh what and a lot of times. It'll be like oh we've been doing this without an index for 10 years and it's been fine, but it just got to the point where oh this like index that is very obvious should be there.

01:26:48.31
Ian Landsman
Right? yeah.

01:26:55.73
Ian Landsman
Yeah depth.

01:26:57.54
Chris Morrell
We finally need to add it and then it's like man now we have to wait to add an index to you know a couple million rows or whatever. But.

01:27:00.50
Ian Landsman
Or I Love Indexes Indexes are the best. So glorious magical.

01:27:06.78
Chris Morrell
Yeah, yeah, yeah, it it that was a big that was a big shift was just like you know the the old school Mysql Mindset versus like thinking and ah like active record style patterns. It was it was hard to it was hard to get into that. But I I feel like I'm finally there I'm finally like I think think about let's make it easy and eloquent first not like.

01:27:25.60
Ian Landsman
And.

01:27:40.14
Chris Morrell
Let's let's join 15 tables together because it's slightly more efficient. You know? Yeah yeah, every once in a while. Yeah, all right? Well I know we've been going for a while is there. Ah.

01:27:40.67
Ian Landsman
Right? Yeah, yeah, yeah, lots of join in back in the day for sure and most of the time. The active record is fine sometimes not but what's the time it is yeah.

01:28:00.11
Chris Morrell
Anything else that you wanted to touch on before we call it.

01:28:03.94
Ian Landsman
Um, no I feel like we we covered a lot of stuff here caching old Apps eloquent.

01:28:07.94
Chris Morrell
Yeah, we we we didn't Ah, we didn't get Yeah I'm still not ah, convinced that you've given us your your most ah most ah terrifying cash situation but you know.

01:28:21.43
Ian Landsman
I don't think I'm not there yet see I had to come back on like in ah you know six months or eight months and we can revisit because see the thing is I'm only I'm not really to the um I'm not full on into the ticketing yet. And the ticketing is where like all the crazy caching is and so I'm still kind of more on the edges of like the tenants and the users and like lists of labels and like just this like ancillary stuff that's more straightforward um whereas the the ticketing is not straightforward and has a lot of.

01:28:37.49
Chris Morrell
Um.

01:28:44.69
Chris Morrell
Sure.

01:28:54.44
Ian Landsman
Complexity and just look like you know we're dealing with email and there's millions and millions and millions of emails. There'll be billions of emails in here and so like that's where it's Goingnna have to take a lot of thinking through and what what corners are we goingnna cut and how we gonna manage that. Um so I'm not all the way there.

01:29:00.47
Chris Morrell
Right.

01:29:12.92
Ian Landsman
I mean I think I have a good idea. We talked about a little bit here but you know when the rubber meets the road um be a little bit different. So I'll have to come back on and talk about what we actually ended up doing there. Um with the caching stuff. Yeah.

01:29:18.26
Chris Morrell
Sure.

01:29:25.40
Chris Morrell
Yeah, we'll see next February you'll be like yeah we ripped it all out.

01:29:32.46
Ian Landsman
Totally totally totally possible. Um, yeah, and how much like I never had a production app on single store. So seeing what that gives us like being a column store database. There's some other advantages there. So there might be things that we don't need to cache as heavily in certain scenarios and other things we do and.

01:29:38.83
Chris Morrell
Yeah.

01:29:50.32
Chris Morrell
Right.

01:29:51.77
Ian Landsman
It also does tons of other stuff. It's way more advanced than Mysql for certain things. So assuming we stay with that even like right now I love it. But who knows like when we get pushed a little bit farther in maybe I'll be like oh you know what don't love it anymore. But ah so far I love it. It's really good, but.

01:30:07.67
Chris Morrell
Yeah, um I I actually did have 1 other thing that I I wanted to to ask you about but we can we can leave it if you if you want to call it? Yeah um, what's ah, what's your what's your approach to you know.

01:30:11.70
Ian Landsman
We'll see. Yeah, no I got time go for it.

01:30:25.32
Chris Morrell
Quarry building like the this the dynamic. Ah you know giving your users all that reporting capability like is that all homegrown or using some library that handles it for you or you know that's something that we've had to.

01:30:39.41
Ian Landsman
Yes.

01:30:43.44
Chris Morrell
Had to deal a lot with as well. Um, and it's an interesting space.

01:30:44.67
Ian Landsman
Yeah, so it is like and I feel like there's not that many apps in some ways that really have to do it So there's not a lot of really great libraries out there I mean if you have some I would love to hear them? Um, but you know in in current help spot like.

01:30:52.22
Chris Morrell
So.

01:30:59.11
Chris Morrell
So now.

01:31:04.45
Ian Landsman
All homegrown like some crazy classes that do a lot of crazy stuff to make it all work. Um, but and in our own Yeah custom coded up like ah you know the builder Ui stuff to build it and subgroup it and all that stuff. Um.

01:31:06.10
Chris Morrell
Mm.

01:31:13.75
Chris Morrell
Just like a tangled mess of Jquery. Yeah.

01:31:22.83
Ian Landsman
And then Aaron Francis did have a tool for it. But that's kind of been put put to the wayside I am planning on looking at what he did ah and I might steal some things from that for sure. Um, but it depends a little bit. You know we have a lot of complexity right now around.

01:31:23.20
Chris Morrell
I know.

01:31:41.29
Ian Landsman
Custom fields and so there's just a lot of logic there because like people can build custom fields and in current helpspot actually makes a column in the database and so you know there's a table that tracks all the columns that have been made and then we got to loop through those and like whatever like.

01:31:48.60
Chris Morrell
But.

01:31:58.41
Ian Landsman
Every query like that deals with custom fields has this whole extra kind of path of stuff to deal with um and in the new help spott ah di'll. All just be in Json so that's going to be interesting. Um.

01:32:13.00
Chris Morrell
Um.

01:32:16.15
Ian Landsman
So it makes a lot of things a lot Simpler theoretically? Ah, but there will be different tradeoffs there of like performance. So yeah, there basically isn't indexing essentially in a multi-tenant solution. Um.

01:32:17.16
Chris Morrell
Sure. Yeah, indexing is a lot trickier. There. Yeah, yeah.

01:32:33.53
Ian Landsman
Because basically like you like you could build a like a ah index for it. But then like everybody's use case is different so you can't actually have enough indexes for all that or you could make it pre-defined like hey we're going only going to have 50 custom fields and.

01:32:50.40
Chris Morrell
Yep.

01:32:50.91
Ian Landsman
This these tenor date and these 10 are strings and these 10 are ints and whatever it can I have so many yeah that would be I mean that's not totally off the table yet. Um and you could just have a really and that way I wouldn't even use Json I would actually just use because basically behind the scenes it just.

01:32:54.84
Chris Morrell
That's what Ah, that's what intercom does you can only have like they just have a hard limit. Yeah.

01:33:06.30
Chris Morrell
Yeah, yeah.

01:33:09.53
Ian Landsman
Makes columns so you just have a really wide table. Um, which single store does actually have some special support for and compressing when you have really wide tables like that really wide tables are super annoying like we do have them in current help spot and they like use a lot of extra space and.

01:33:10.98
Chris Morrell
Yeah, yeah.

01:33:16.79
Chris Morrell
Next.

01:33:25.17
Ian Landsman
Hurts performance and different ways and stuff and then you also have different limits of like how many indexes you can actually have with current helpspot like you can make ah as many custom fields as the table allows in Width which is like 300 columns usually roughly depending on how big they are in different things but it only allows you have 64 indexes

01:33:28.31
Chris Morrell
Yeah, yeah.

01:33:39.70
Chris Morrell
Sure.

01:33:44.16
Ian Landsman
So you end up having to like you know deal with that that um, you can't just assign Index Always yeah, which we don't know the priorities so that becomes own thing. Um, so right now.

01:33:48.50
Chris Morrell
You Yeah, you have to prioritize things. Yeah yeah, another another good use case for events sourcing just going to say well. Ah I mean I think.

01:34:00.10
Ian Landsman
Is yeah how is that how is a bad source of the solution to this.

01:34:05.41
Chris Morrell
The advantage there is um, you can essentially just decide after the fact like okay this this tenant. Ah you know is hitting is hitting these specific like data points all the time you just you just.

01:34:20.14
Ian Landsman
Like just tracking it. Yeah, right? Yeah, and.

01:34:25.32
Chris Morrell
Build a bespoke custom custom table for them and like it just it kind of comes for free. You know like you still have to do the work but like you could automate a lot of that where if you had Dc decent analytics. Yeah, you could just have like ah a monthly or weekly or whatever job. Essentially just rebuilds ah you know, custom tables for yeah for all like your active tenants and you could even make that like ah you know a tiered feature where it's like you get regular performance. But if you need like enhanced performance. Yeah, like.

01:34:47.18
Ian Landsman
But indexes on a table or something.

01:35:00.42
Ian Landsman
Optimize performance. Yeah well for next generation healthspot. It's gonna be a more traditional multi-tenant situation where everybody's in one database with Tenant Ids and I probably wouldn't want to like break it out that way there. So we're just gonna make it very performant is my plan.

01:35:02.93
Chris Morrell
Um.

01:35:10.23
Chris Morrell
Um.

01:35:13.24
Chris Morrell
Yeah, it's an interesting problem now.

01:35:17.37
Ian Landsman
But we could do that? Um, we could do the wide. The wide column is like kind of the other option of course and then just how I think that's how like fathom does it? Also um I don't love it because it's like creates these really artificial limits of like how many custom fields you can have.

01:35:23.30
Chris Morrell
Yeah, yeah.

01:35:34.36
Chris Morrell
Um, it it sucks I mean that it's It's a problem. Yeah, it's a problem I have with intercom is like you can't because then the the problem is you can't just like throw all your data at it and figure it out later right? You have to like.

01:35:36.32
Ian Landsman
Obviously of different types. Yeah, they yeah right? Oh I'm out of my 10 ints like that's it right? Ah yeah.

01:35:49.63
Ian Landsman
Right? Yeah, yeah, exactly that's where I don't like that So that's where the help help spot has quite a few customers that are like.

01:35:53.72
Chris Morrell
Be really thoughtful about what data you have to know now what data am I going to need in six months for analysis. You know, which sucks? yeah.

01:36:04.85
Ian Landsman
We put an unreasonable number of custom fields in there and we're just storing everything in this way if we ever need to report on any slicer dice. You know thing with support. We have all the data in there and it's all there. So yeah, exactly like and there is tradeoffs there I mean it's like the Json Column solve it for the most part.

01:36:08.53
Chris Morrell
Um, yeah.

01:36:14.96
Chris Morrell
Um, yeah.

01:36:23.42
Ian Landsman
Um, especially with like single store as very fast Json Lookups and things. It's highly optimized. So I think for most customers it's going to be fine because obviously we're going to limit the scope with other conditions. Um, so it's actually not looking.

01:36:29.99
Chris Morrell
Um, that's cool.

01:36:39.85
Ian Landsman
Not table scanning the Json Hopefully most of the time but there will be some elements there where that could happen so we'll we'll see but it's it's been quite fast and testing. So hopefully that works. Oh yeah, that's happening for sure.

01:36:43.33
Chris Morrell
Um.

01:36:52.55
Chris Morrell
Now Would you ever consider like pre-computing values. Um, yeah, yeah, because that's that's um, ah okay, yeah.

01:36:58.88
Ian Landsman
Yeah, that's 7 that doesn't happen in current help spot. But that's like part of the denormalization stuff that we're gonna do in in next generation help spot. Yeah, you just kind of have to.

01:37:09.84
Chris Morrell
So We use Symphony Expression Language have you ever heard of that. It's ah it's like ah it's basically just like a simple query language. Um that compiles down to php. Um and it's been like. I don't I don't know that it would work for your use case but we have a pretty sophisticated custom query Builder situation. Um, in our case, you know because we're we're essentially ah I like to say the core thing that we do is answer whether you did something like.

01:37:44.50
Ian Landsman
I I.

01:37:47.70
Chris Morrell
Certification organization but like to be certified. You may have passed 3 these 3 courses and like got a certificate from this other like ah you know, live training class or taking these. 5 other like more intensive courses and passed an exam or like if you join before 2015 and you took this other course. Yeah there's like there's potentially like 15 paths. Um, so we need to have this like really really complicated um logic around. Ah.

01:38:04.00
Ian Landsman
Right. Right? right.

01:38:21.60
Chris Morrell
What constitutes sort of being meeting the requirements of a given program and so we have this really really sophisticated program requirements system that comes with that that we built like ah a builder ui for and.

01:38:21.65
Ian Landsman
Um, ah.

01:38:35.10
Ian Landsman
And.

01:38:38.36
Chris Morrell
You know I'm just thinking about your use case I'm not sure but what we've been able to do is essentially you know we're solving it kind of like going back to the caching. We know what values might change to change someone's eligibility for a given program.

01:38:49.33
Ian Landsman
M.

01:38:55.49
Ian Landsman
M.

01:38:57.75
Chris Morrell
Um, so when any of those values change. We just queue up a job that pulls in all the potential data that they need runs it through the expression and caches the value and that way you know.

01:39:05.66
Ian Landsman
Pick your head out. Yeah.

01:39:16.94
Chris Morrell
At the database level. It's as simple as like there's programs and there's enrollments right? and like an enroll you're either your enrollment is either like in progress or complete. Um, so to know if someone actually like has met the requirements of a program we can just look do they have a complete enrollment.

01:39:18.62
Ian Landsman
Right? You just have a flag or whatever. Yeah.

01:39:32.44
Ian Landsman
I.

01:39:33.67
Chris Morrell
Right? We don't have to do any. We. Don't do any work whatsoever and instead we've just like invested in making sure that you know any time someone completes a course we look up all the programs that rely on that course's data and find everyone who has like a incomplete enrollment for that program and we just. ue up a bunch of jobs to reevaluate their their status. It's not real time right? It's pretty close to real time. It's usually within thirty 30 seconds inute because we just spawn a bunch of jobs and and deal with them in parallel. But um, it's a it's been a really powerful approach.

01:39:56.17
Ian Landsman
Yep. Right? yeah.

01:40:12.72
Chris Morrell
Um, and using something that's like ah a full-blown language but is um, you know but is is scoped enough that you're not like dealing with ah security concerns or like other major issues. Yeah, so it's ah it's a Symphony Framework package. It's called Expression language.

01:40:24.50
Ian Landsman
Is this part of Symphony the framework or it just happens to be called company. Yeah, right, not to check that out.

01:40:32.60
Chris Morrell
I Think they use it for um configuration if you want um and they use it in a few places but ah, it's It's been like a pretty. It's It's been a pretty good tool for us it definitely. There are tradeoffs and and the tradeoffs are really.

01:40:48.42
Ian Landsman
Um, right.

01:40:51.37
Chris Morrell
I didn't even think about it but they really fall into that same category of caching right? like cache in validation is hard and this is essentially caching values. So it's it's ah yeah, um, but it's it's worth looking into. It's It's pretty cool. Um.

01:40:56.58
Ian Landsman
Right? right? You're caching their status.

01:41:06.26
Ian Landsman
Yeah I'll check that out I haven't even heard of it. Um, yeah I think we'll be doing some I don't know see the weird thing for us is like there are some things that we're definitely going to handle like that like the status of a ticket we do differently now. But it's going to be more along those lines where like we know its status based on different actions and we can just move it.

01:41:25.92
Chris Morrell
Yeah.

01:41:26.13
Ian Landsman
Um, and things like that. But the custom fields especially are trickier because we just don't have any context for what they they could store anything right? and we don't know what it means to the organization and things like that. So it apply. We have to be more abstract than that but that is along the lines of my plan. It's going to be depend a little bit when we actually get in there and like throw a bunch of data in there like a ridiculous amount of data and see is the Json stuff fast enough that we can just rely on it or should we break it out and like de-normalize it into because single store being column store is like super fast if you break it out in tables.

01:41:49.90
Chris Morrell
Yeah, yeah.

01:42:03.20
Ian Landsman
Ah, aggregating things. Um, so I mean that's how a lot of people just use it actually they actually take it out of normal relational databases and break it up and put it in a column store database so that you can run. Okay, we have 20000000000 rows like let's aggregate those. Whatever our aggregate reporting needs are very quickly even over billions and billions of rows. Um, so we might do some stuff there. The reporting stuff I'm kind of waiting that's going to be more towards the end when I get in there but that should actually have some cool overengineering type stuff. Potentially um, when we get there. It's a little tricky because like helpspo isn't.

01:42:21.62
Chris Morrell
Yeah, yeah.

01:42:39.72
Ian Landsman
Intercom ah in terms of scale. So like most of our customers are you know 1 person to 200 agents is kind of like our range. Um and you know tens of support tickets to you know, maybe? ah.

01:42:49.30
Chris Morrell
Ah.

01:42:58.89
Ian Landsman
Couple million a year or maybe ten million a year at the the high side so you know it's not like crazy numbers in any 1 tenant where you might have like a big support org that's doing 50000000 or something like that. Um, but still it adds up over years and years it does add up to a lot. For the big customers. So we thought the account for that somewhat somewhat so we'll see but it's gonna be It's gonna be interesting. Definitely some interesting. It's it's just interesting to like approach knowing so much about the problem space and then approaching it is like.

01:43:19.65
Chris Morrell
Yeah.

01:43:30.62
Chris Morrell
Yeah, yeah.

01:43:32.92
Ian Landsman
Really interesting and somewhat debilitating at times because you're like oh man I Know how hard this is I know that there's no good answer for this anyway. So which of these not optimal answers am I going to choose and then like that grinds you down because you're like well what is the best way to do this nowadays. It's like like custom field is like the perfect example, there's still no best way to do it.

01:43:37.94
Chris Morrell
Ah, yesst, yes.

01:43:46.29
Chris Morrell
So yeah, yep, yeah.

01:43:51.32
Ian Landsman
Like there's still no like everybody does custom fields this way. This is like how it works. No, it's like every all the 3 or 4 versions of custom fields are all have horrible terrible tradeoffs and that was true. Twenty years ago when it took me forever to figure out what I wanted to do and it's still true now it's super annoying that there's not like an answer yet. But.

01:44:04.47
Chris Morrell
Yeah, yeah out. Yeah.

01:44:10.69
Ian Landsman
You know we'll we'll work it out. Everything's faster now which does help to some degree so you can kind of get away with some of these Json like solutions a little more easily than twenty years ago you could not have done that because they're just not for the scale help spot operates that because it just would be destroy the database right? Ah, but ah, but now it's not not as true. But.

01:44:15.64
Chris Morrell
Great.

01:44:22.50
Chris Morrell
Right? You just? yeah yeah I mean like you you also run into that problem that that you know enough you you're not.

01:44:29.83
Ian Landsman
Anyway.

01:44:40.42
Chris Morrell
Inexperienced enough to not know you can't do it like there is that I mean there are things I think that if I I just wouldn't have even tried to build now. But but when I built them I didn't know how hard it was going to be right.

01:44:41.42
Ian Landsman
Right? right.

01:44:49.61
Ian Landsman
Right? yeah.

01:44:56.15
Ian Landsman
Right? yeah.

01:44:59.83
Chris Morrell
And I just made it work and in some ways I'm dealing with the repercussions of that still to this day but in other ways that was what helped make us successful. You know, like there is a little bit of a risk in knowing too much.

01:45:08.47
Ian Landsman
Yeah, there is there is like we and way back. So I originally learned the program and was using Microsoft Sql server and.

01:45:19.84
Chris Morrell
Okay.

01:45:23.64
Ian Landsman
Which even back then was you know of an advanced database and then Mysql is way less advanced and so but obviously the php space just like Mysql is what everybody uses so fine. That's what I was using but like I didn't really understand the difference that like my Isam and like its limitations and it didn't really.

01:45:39.25
Chris Morrell
So.

01:45:42.27
Ian Landsman
Ah, you know all the limitations of my iam and locking and different ways of how it works compared to like a more sophisticated database and or in in odb that we have now which can do atomic locking and things like that like how my Sam would just lock the whole table when it had the right? So like very early on like I hit.

01:45:59.96
Chris Morrell
Yep.

01:46:02.22
Ian Landsman
Crazy stuff with that because I was like oh like a bigger install like has all these conflicts and timeouts and bad stuff's happening So how to like optimize stuff to get through all that but it's like yeah, it's like you don't even know like here's this huge part of this platform. It's going to have all these impacts like didn't even know it was a issue and like.

01:46:19.18
Chris Morrell
Yeah.

01:46:21.66
Ian Landsman
But customers in there that are like ah it's broken and you're like oh man it is broken. That's the bummer.

01:46:27.36
Chris Morrell
I we we only switched to n d b and know you know we started switching to n d b around when we switched to when we introduced larall and yeah UTFEight everything was wasn't even latin one. It was ah you know it.

01:46:31.31
Ian Landsman
Recently right? Yeah, that was a big change for us.

01:46:44.69
Chris Morrell
There was like some other ah latin one type it was like CP 2 54 some some weird. Ah yeah character encoding. That's like it. It looks like ascii it smells like ascii but it's not quite um.

01:46:44.87
Ian Landsman
Sounds kind of familiar. Yeah, right something weird. Yeah.

01:46:58.80
Ian Landsman
That was the first project chris fida worked on first was like I had him on this like deaf March of like his whole like like it was like his whole first year was like rebuild like to build the installer an updater that would take an on-premise help spot install and.

01:47:02.43
Chris Morrell
Tough.

01:47:17.16
Ian Landsman
Convert it to utf 8 which like based like we had so rebuild every table like build a whole new database rebuild every table in utf eight and then move all that data between 1 to the other and convert it and fix all the weird stuff in between when somebody had non-latin 1 characters in the latin one table and all that stuff like was it was a bear.

01:47:19.86
Chris Morrell
Yeah.

01:47:30.77
Chris Morrell
Um, yes, yeah, doing we did that all in this is the only time that I've ever used ah like Mysql functions but doing all of that inside of a Mysql function was.

01:47:41.16
Ian Landsman
M.

01:47:47.98
Ian Landsman
Just saying and.

01:47:49.10
Chris Morrell
Magical it was. It was so easy. It was essentially just like it's just you just write an infinite loop of like it's essentially like select insert into select from with like a bunch of stuff before and after to like manage the state.

01:48:01.65
Ian Landsman
My.

01:48:08.60
Chris Morrell
But because it's happening all in database like a lot of the issues just go away and you can just essentially just have it run in a giant while loop. That's just like as long as there's records that you haven't migrated over yet just keep on going and then the moment there are records that there are no records that haven't been migrated over.

01:48:08.83
Ian Landsman
Um.

01:48:19.27
Ian Landsman
Crank it out. Yeah.

01:48:28.00
Chris Morrell
Swap the table names you know and like it's almost like a hot hot switchover it was it was kind of an amazing because we were able to switch to NODB and UTFEight it took a lot of trial and error to get there. Um, yeah, it.

01:48:28.00
Ian Landsman
Yeah, okay.

01:48:39.41
Ian Landsman
All in ones swoop. That's what we did to I think was also in to b at the same time. Um, yeah was and it was um even simplified because we actually support three database platforms so sql server. Postgres and Mysql. So there was like a bunch of stuff with that. Yeah, it's totally there Sas is the Sas is the way. Um, we still saw on-premise to some hand hand you know there's still whatever 20% of customers that are doing on-premise um new customers. But.

01:48:55.40
Chris Morrell
Um, yeah, that's a nightmare. Yes.

01:49:09.42
Chris Morrell
Okay, that's a lot. Yeah, that's huh Yeah right.

01:49:12.60
Ian Landsman
So it's still a thing. Yeah, that's still still a chunk well because if you want to like a help desk out that's on-premise like your options are becoming more and more limited there actually aren't as many like nobody's starting new ones and a lot of the old ones have converted to being Sas only and things like that. So right.

01:49:27.15
Chris Morrell
Right? Well and especially like a decent 1 right? like? yeah yeah, yeah.

01:49:32.50
Ian Landsman
But yeah, with that's actually supported I mean there's some open source things that have no support and things like that. But that's not usually that appealing to not for and you know it's like open source is very weird. It's one of these things where like everybody all these companies use open source infrastructure.

01:49:45.90
Chris Morrell
Yeah, for sure.

01:49:47.90
Ian Landsman
Um, or whether it's larabelll or obviously like lower level operating systems and everything else and there's either no support or not much support and but it's like everybody accepts that but with endus apps ah bounds not really true like there's not. For like ah, any type sizable organization like nobody wants to be like we're going to go with the open source crm with no support and like we're just going to that's what we're going to do like nobody wants to put their job on the line with that. So. It's just not a thing that happens like at the very least it's got to be 1 that's like yes, it's open source. But.

01:50:07.86
Chris Morrell
Right? right.

01:50:19.43
Ian Landsman
But you you can buy support from the creators or a company that just provides support around that platform like sugar crm or whatever. But ah yeah, so.

01:50:19.53
Chris Morrell
First Yeah yeah.

01:50:27.77
Chris Morrell
Um, yeah I mean that and that has changed a lot I feel like um, you know we used to run like Pbb and V Bolton and we ran you know all these different either open source or or ah or proprietary but just like.

01:50:38.49
Ian Landsman
Right.

01:50:46.97
Ian Landsman
Mostly unsupported, right? Yeah right? right? Yeah, it's onpremise.

01:50:47.71
Chris Morrell
You know it's It's essentially just a script that you throw on your server and now it's your problem. Um, you can call that on Promiseise I don't know that's exactly right? Um, but ah yeah, over the years. It's just like as sass has just eaten everything up.

01:51:06.24
Ian Landsman
There.

01:51:07.61
Chris Morrell
And you know and as as we as an organization have grown. It's just like I don't want any of those headaches right? like the fewer things that I can deal with the better I Yeah right right? Yeah, yeah, we.

01:51:12.55
Ian Landsman
Nobody wants to headaches. You got plenty of headaches as it is you don't need to be responsible for like the Bulletin board script that gets hacked or crashes or whatever. Yeah.

01:51:25.90
Chris Morrell
We pay discourse to just deal with it for us and that's fine I don't love discourse. But I love discourse a lot better than managing V Bolton I'll tell you that. Ah yeah, it's not good I mean there's not.

01:51:29.11
Ian Landsman
No, no, no right? Ah yeah, we use discourse for a while. It's It's not bad, but it does little weird and samaras. Yeah.

01:51:44.38
Chris Morrell
That's that's one of those spaces I Wish there was a really really really good Bulletin board. Um, you know there's ah, there's that water ah waterhole which is a lairravee based ah lave based ah forum. Ah.

01:51:48.16
Ian Landsman
The yeah.

01:51:54.87
Ian Landsman
I Don't know it though. Interesting didn't even know about it.

01:52:03.22
Chris Morrell
It's it's interesting. It's and like it's it's built by someone who has like ah a pretty strong background in ah prior I can't remember what it's It's not one that I had ever ever used, but it was a relatively popular piece of software and they went out I think tried.

01:52:15.82
Ian Landsman
M.

01:52:22.77
Chris Morrell
And and co started a product called flare. Um, which still exists too. But it's like an open source Pb ah Forum software that uses a lot of the like illuminate components but is not a larall app and I think that he at some point just.

01:52:30.10
Ian Landsman
Um.

01:52:40.14
Ian Landsman
Ah.

01:52:42.72
Chris Morrell
Decided like actually I'm just going to build a full-blown like lairvillell package that you can just install in an existing app. Um, it looks it looks promising but I'm never going to use it now now the the migration I've migrated.

01:52:49.38
Ian Landsman
Right? You're not gonna bother switching over like that's that's heavy.

01:53:02.32
Chris Morrell
Hundreds of thousands of ah posts ah multiple times between p two b to v bolton and then v boltin to discourse and I hope to never do it ever again? Yeah yeah, luckily the the discourse one was mostly yeah.

01:53:10.76
Ian Landsman
Those days are done. Yeah, that's not a fun migration different formats and was it that it was figured out. Yeah, that's good.

01:53:21.48
Chris Morrell
Mostly their problem. But yeah, we had to we still had to do we still had to do a surprising amount I think that's actually one of the very first things that Daniel worked on when he started at internet. He was basically migrating ah thousands and thousands and thousands of old.

01:53:32.00
Ian Landsman
M.

01:53:40.99
Chris Morrell
Ph bbb posts that like still existed in a table somewhere to back like ah Backfi the discourse. Yeah ah what a nightmare real world stuff.

01:53:45.27
Ian Landsman
Right? Dig him up and yeah, that's funny. It's real world stuff. That's what happens in a real business times you gotta go down into the mines and do what you gotta do.

01:53:57.00
Chris Morrell
Yeah, you got it, You just get to live with it. Yeah, but that's fun I know I I love I Love um I Love modern development. You know it's fun to think about back in the day. But.

01:54:11.33
Ian Landsman
Go No, we come a long way.

01:54:16.26
Chris Morrell
Just the the tooling that we have the the fact that yeah like you can just say yeah things are fast enough I don't have to worry about it. You know, like yeah sure a couple million records is nothing now right? Ah and ah.

01:54:23.17
Ian Landsman
Yeah, yeah. My.

01:54:33.64
Chris Morrell
Just all the all the tooling that we have I mean especially in the larall ecosystem. it's it's amazing it really is like it really does so I wouldn't I wouldn't give it up all the all the testing and Ci and it's it's great, but ah, it's not always.

01:54:36.80
Ian Landsman
Yeah, makes a huge difference.

01:54:48.50
Ian Landsman
Now.

01:54:52.91
Chris Morrell
It's not always exactly like that.

01:54:56.20
Ian Landsman
There's still some weirdness that creeps in here and there users right? You're why not that successful really? That's the that's the real takeaway.

01:54:56.76
Chris Morrell
Yes, indeed. Yeah, if if you don't have weirdness that's crept in you haven't been around long enough.

01:55:08.50
Chris Morrell
Yeah, there you go all right? Well we're we're close to 2 hours ah this has been great. Um I had I had a really good time. Thanks for thanks for hanging out.

01:55:10.91
Ian Landsman
There we go I think we gave everybody some listening.

01:55:20.75
Ian Landsman
Yeah, thanks for having me on I'll come back. Ah in the future and we can get into some ah ugliness that actually ships. Yeah yeah I mean thanks a lot.

01:55:26.57
Chris Morrell
Yeah I I would love that the follow up the follow up. Let's ah we'll wait six months and see where you are nice. All right. Thanks.

01:55:47.93
Chris Morrell
I now that I'm doing video I still it's like you know we used to just Bob our heads while we were listening to altra music. But now people are watching us bobberheads and now they have to listen to this all right.

01:55:59.15
Ian Landsman
Ah.