Game Economist Cast
What does the new wave of open economies mean for monetization? Will negative externalities overcome cosmetics economies in the long run? What exactly does a game economist do?
Game Economist Cast is a roundtable discussion of the latest developments in mobile, HD, and crypto games through a bunch of people figuring it out using the economic tool kit.
Game Economist Cast
E19: Why Doesn't Apple or Steam Use Regional Pricing? (w/Bill Grosso)
Price discrimination is the economist "duh," yet few firms engage in the *welfare* enhancing practice. What's with that? Dr. Bill Grosso, CEO of Game Data Pros, joins the crew to tell us that sometimes it's just about more columns in the database... We cover his efforts to build price personalization at scale, community pushback, the pricing power of brands, and the best things about game economists.
[1] A two-armed bandit theory of market pricing
My teacher, like for some reason, was super into Mathematica, which is their software, and so I was like, oh well, take you kids to the Mathematica conference.
Speaker 2:I had a professor at Purdue try to teach me Mathematica. I was like dude, this is not going to happen. I refuse. It's horrible, Like trying to use Matlab.
Speaker 1:It was like the first programming language I really used and so like I was like super into it and then after I stopped using it for a year I never touched it again.
Speaker 2:You tried something else. That's an actual programming language.
Speaker 1:You can make some nice visualizations with it, though.
Speaker 3:Yeah, but Gigi plots better, to be honest.
Speaker 4:Oh, I love it. You're in my book.
Speaker 2:We cannot start the programming language wars just yet. We got to wait till a couple of minutes into the episode. Get the introduction in first before we start fighting about R versus Python versus Stata.
Speaker 3:Let's start with utility. I don't understand what it even means.
Speaker 2:Everybody has some kind of utils in their head that they're calibrated.
Speaker 3:There's hardly anything that hasn't been used for money.
Speaker 1:In fact, there may be a fundamental problem in modeling. What a model?
Speaker 4:Here we are, episode 19. The interviews keep on rolling. I don't know how we keep doing this. Guess after guest, and we're joined today by Bill Grosso, the CEO of Game Data Pros, formerly Scientific Revenue, and I also started to come across this two fish organization, which I'd love to learn more about and what your role was in that. But Scientific Revenue is something I followed for a really long time, or at least the story of Scientific Revenue. What would you describe as the thesis of Scientific Revenue and what you were trying to achieve with that company?
Speaker 3:Great question and first of all, thanks for having me on. I'm super excited to be here. Following Dr Ayo is a tough act, but I'm really excited.
Speaker 3:And actually the answer to that question probably goes back a little bit to two fish. So two fish was a payments processor. At the end of the day, it was microtransactions with about 170 or so payment methods by the end of it, so that desktop games could take local currencies and local payment, not just credit cards, not just bank transfers, but the entire gamut of stuff. We sold it. We sold two fish to a company called Live Gamer. It was an interesting thing to do. We had amazingly good technology at two fish. We built out a high end transactional engine based on OLAP data cubes. If you remember OLAP, we had really good analytics for the time, but we had no customers. And Live Gamer was in the opposite situation. Mitch Davis and Andy Snyder were running it. They had great industry connections. They had people who wanted a payments processor and a microtransactions platform, but their technology was fairly rudimentary and, ok, let's do this thing Right. We merged the companies. That achieved some level of success. But around 2012, were there abouts? What happened was you could see the writing on the wall Microsoft and Sony weren't going to let us process transactions on the consoles, google and Apple weren't really the latest process transactions on the phones, and payment wall was around. Exala was around. Even then, it was a low margin business and we had a lot of discussions about that and what I said was what time was 2012? Bill, 2012.
Speaker 3:And my point of view was so we had a successful microtransactions based payments processing system with OLAP data cubes. We weren't yet at the era of machine learning, we were at the era of big data. So, like now, we're in the era of AI. Before AI was machine learning, before that was big data. And I was sitting there going.
Speaker 3:All these companies that are using Live Gamer to do their microtransactions because they can't figure out how to interact with like local payment instruments, probably can't figure out how to price their goods effectively either. And I'm a former mathematician, my undergraduate degrees in economics. I now understand the world's payments infrastructure and I'm getting kicked out of the industry I'm in by Google and Apple and Microsoft and Sony. I'm going to go into the dynamic pricing industry and just build the layer on top of Live Gamer that does pricing of goods and services based on big data, which was the buzzword at the time, and so that was the transition into how scientific revenue started. It was just. There appears to be this big open question of what should I charge, and that the gaming industry wasn't terribly adept at figuring out and the business I was in was going down this low margin, sub profitable path.
Speaker 4:What do you see is the difference between asking what I should charge versus dynamic pricing? Is there a difference there? What is the big dynamic system come from?
Speaker 3:So there's a pretty big difference, because you can ask the question what should I charge? And you can say I'm setting the price for the entire population and the question is what should I charge? You could say that I'm setting the price once, when I think about dynamic pricing and I think about dynamic pricing in digital entertainment I think of it as more the prices may change over time and they may be personalized over time, and what should I charge is foundational question, but it's not necessarily you can answer what should I charge without going down a dynamic path and without going down a personalized path? And so the dynamic pricing part is an extension of the what should I charge?
Speaker 2:Bill, did you have an industry that you use? So when I hear as like a mental model, when I hear dynamic pricing, I think like those prices are changing all the time. From minute to minute, different IP addresses get different prices. The models that they use, they are incredibly sophisticated. Did you have you were trying to apply this new technique to a relatively new industry? Did you have something that you were using as a baseline or were you just winging it? Oh, absolutely.
Speaker 3:Absolutely winging it. There's not a lot of prior art, right, if you look at what airlines do, and it's lots of verticals, right. It's not yield optimization or revenue management is a new idea, but it's a very different set of problems and their solutions are all. Not only is it a different set of problems, but their solutions very much reflect the time in which they were conceived. Airlines started doing revenue management in the seventies. Not a lot of data that your people at that point.
Speaker 3:And in fact and I don't know if this is true anymore but I had conversations with airlines in like 2016 and they were like well, I didn't even claim we can only sell seats for seven prices, because that's the. We have columns in the database and there's and I was like what are you serious? It might not be seven, but it's that many, some finite number of columns. And I was like, ok, and then they were like and we preallocate them years in advance, because some percentage of our traffic is sold through vacation retailers make this bundle and price it and then sell it to someone who's going to travel nine months later, and so there's like an 18 to 24 month lead time on some of the airline prices because you have to make a commitment to your travel partner, who then has to create a bundle and assemble everything and then ship it. And so you're like, ok, so you only have a small number of columns and half of them are preallocated to your business partners who have built in lead time.
Speaker 3:And I was like, ok, and I talked to a bunch of airlines in the 2016 timeframe and I was just astonished that one of the things they did with the unbundling is OK, we're going to charge you for the extra suitcase, we're going to charge you for the food, we're going to try and sell you a hotel room when you go through the loyalty program. And that was all driven by the fact, or at least partially driven by the fact, that they didn't actually have a lot of freedom to experiment with pricing, which is an astonishing thing when you say super sophisticated thinking in 1970, when the idea of yield management was created. Now it's got 50 years of bad software underneath it, and not bad for the time. Software embodies the limitations that are possible when you write it.
Speaker 2:Speaking of software and going back to scientific revenue, would you say that it was a? It was a product driven business, or was it a consultancy? Were you just consulting with these organizations and saying here's what she should do, or were you building products back then?
Speaker 3:We were trying to build a product. The Genesis was, as I said, we wanted to, or the original vision was a dynamic pricing product for video games, in particular, for mobile games, and the core idea was we wrote an SDK, you plug the SDK into the game and that was to solve all the data issues, with every game company recording things slightly differently. So we have Trusted data and now we're going to apply a dynamic pricing engine that's going to basically figure out the right price points, and that's really interesting in and of itself, because and you don't have infinite flexibility on the app stores either right, so you wind up pre-creating a bunch of bundles because that's the and then choosing which bundles you expose to each users. And the idea was we could automate that, and this is a colossally arrogant thing to assume we could do, if you're like keeping scores, okay, raised venture funds to convince an industry that it needed dynamic pricing, while also inventing the underlying technology for doing Direct pricing and also, to a certain degree, inventing the theory involved.
Speaker 2:I'm assuming you didn't get funding from Sequoia Capital. They're like antithesis. Is you like don't try to build a business around a market that doesn't exist? I like you can't build a market in all, and they're right, honestly and truly, they are absolutely right.
Speaker 3:When you look at Execution risk, it's okay. And there was this great paper from the early 70s by a guy named Ross tile at Princeton, like a two-arm bandit theory of pricing. It's a great paper and but begins with, like Economic modern economics, lacks a good theory of how businesses should set their prices. It's like in the opening paragraph is some sentence like that and you're like, yeah, that's true, we have utility functions, we have marginal utility, we have all these curves in the intersect, but where is that intersection? How should I set my prices as a business? I don't really know.
Speaker 3:And the proposal was to use essentially multi arm bandits and test, which is not a thing you can actually do in 1970. So it's entirely a theoretical paper, but it's an interesting thought and so it's scientific revenue. I'm gonna convince the gaming industry it needs dynamic pricing. I'm gonna figure out how to do it. We're gonna implement it on top of this big data stuff which, quite honestly, was super immature at the time, and we're gonna do it all in an 18 to 24 month time frame, because you don't get a lot more money from the VCs than that.
Speaker 2:And it was a product. I mean in some respects you were right like it's some of those goals. You're dead on the money, like this dynamic pricing thing. That's all for you play 10 years later.
Speaker 3:I clearly was right. But you can be right and yet have a enormous train wreck of a company what's that line? It's too early is the same as being wrong, pretty much, yeah yeah, and a large part of what we do at GDP Game data pros is built on our knowledge from sign revenue. It's in many ways, a logical continuation of the idea.
Speaker 4:So when I think of dynamic pricing as an economist, my next thought is you're doing price discrimination. It's just like a really nice term for it in the same way that people used to talk about marketing and now we talk about user acquisition and growth people. Is that the right way to think about it? Or is dynamic pricing different from price discrimination, or the theory is different?
Speaker 3:They overlap to a huge degree, and that goes back a little bit to the how much should you charge question, right, you can have flat rate pricing that everyone gets, and then you're not doing price discrimination, but you are doing some elements of optimization and some elements of trying to figure out what the right price is, given that you're going for everybody at the same.
Speaker 3:And I really don't like the term price discrimination, primarily because in the us, we have this idea that discrimination is bad. Anything you apply the word discrimination to automatically becomes bad, and then, like you read the economics literature, that's type two price discrimination. I really wish you hadn't called it that, because now people heard the d-word and now they're not gonna. I'm bad just by definition because of my. Now we aren't, and there and there's lots of regulations in the us around it Like you can't discriminate based on gender or age, or and those are protected categories but, yes, past that, you absolutely are Engaging in price discrimination, or you can be engaging in price discrimination. It usually doesn't go the way people think it goes, though, so I got an enormous amount of negative Feedback and hate mail when I was CEO of scientific revenue.
Speaker 4:I remember this I remember. I remember when, when people talked about price discrimination, they lose their shit. They lose their shit. I'm sitting there as an oculus being like someone's got to defend this guy, someone has got to stand up for this guy. But I remember people go age shit over this and they do just it blows my, and part of it is.
Speaker 3:One of the big complaints that emerged in the emails over and over again was the idea that I was being psychologically exploitative, watching for signs that somebody was addicted to a mobile game and then raising prices on that curse.
Speaker 3:And it turns out not to be the case. There are enough substitutes in the mark that if you raise prices on people to Exorbitant degree, they they find another game, and the data is nowhere near good enough to spot psychological addiction. If you look at game telemetry and then you say I can spot this player is now no, not really. Best you can do is spot that they're in a habit of playing it, but that's a different thing. So what happens in practice is more around things like price skimming and more around discounting around To people who are going to grind, and so the actual impact of price discrimination in gaming Is you lower prices for the people who are unlikely to pay the original pricing, and so you get more people buying, you get more players converting, you make more money, but you haven't actually gone out and engaged in the exploitation of psychologically vulnerable people, and I actually tried to make that case in public a couple times and people just didn't believe me. No, but underneath it you're engaging in discrimination and you must be evil.
Speaker 4:Let me so. That makes a lot of sense to me and when we think about like price discrimination, it can be welfare enhancing right, like we're gonna offer matinee discounts or we're gonna offer discounts to college students. That's welfare enhancing. Everyone is better off. But let's say we find ourselves in a situation where we've identified a customer Does have a higher willingness to play or they have a lower price elasticity.
Speaker 3:Should we raise price? I'd say that depends on who you are. Can you raise prices? Yes, should you? Different question? So I a depends on the product as well.
Speaker 3:Here's a great example I was raising money for a bridge loan for scientific revenue and there was a guy who was going to put money in and he wanted me to meet him. And we actually met in a bar in Vegas. I flew to Vegas to meet essentially the enormously wealthy person who owns own bests, right. And he was like look, I'm a whale. I know I'm a. You know I'm a whale. The game has to know I'm a whale and I would like to pay to win, because the reason I'm here is to win. I like winning games.
Speaker 3:I like why are you telling this is you should raise prices on me and you should have all the companies using scientific revenue when they spot me and they're running a tournament and I'm buying something in a tournament time frame, charge me the same price for my first, first Per. Charge me more for my second purchase. Charge me even more for my third perp. I'm willing to pay for it because I'm here to buy the win. And I was like okay, so that's a really interesting thing.
Speaker 3:You found someone who, essentially, is going to buy it no matter what, and You're advocating raising prices on the people who are too affluent, almost as a social tax, and it makes the tournament fairer because some percentage of those people will drop out of the purchase process after one or two purchases and it increases revenue, and so there's an enormous amount of social good in. And should you do that? Probably. The other hand, when one of I spent a lot of time in Europe when I was running, signed, you know, and I met Dave Nelson once and he like told me what I was doing was wrong and I was like why? Yeah, oh, absolutely.
Speaker 4:We're gonna lead with that headline. I'm already sending Dave like three text messages right now.
Speaker 3:But literally what he said, he like was like this is wrong to do. And I was like why? And he quoted a Swedish proverb at me and I was like I don't speak Swedish, I never trust those.
Speaker 4:Never trust those.
Speaker 3:He translated it was something like people are equal and I was like there's like that means you should chart some the same price. And I was like no, it doesn't. It means people are equal. That's not the same thing as you should. And I use some of your examples, like senior citizens night at the movies. Are we opposed to that? Do? Are we opposed to? When someone lives in a country with much lower gvp, are we opposed to giving them a discount Right? And he was like no, of course not. Of course we should do this for the seniors. Of course we should charge lower prices in in in countries that have a less robust economy. I'm not talking about that.
Speaker 3:I'm talking about what you do, and I was like I have a real problem figuring out what the dividing line is here, it wasn't like he was angry at me, it wasn't like he was like wielding a baseball bat, until I saw this, the true justice of the situation. But it was just a place where I did not understand what he was saying and he was asserting a distinction I really don't follow. It's very hard for me to say we can do these things but not these things, but I can't tell you how to tell them apart.
Speaker 1:I think a lot of the most price discrimination, as you're describing, is actually progressive. It's charging people with less willingness to pay. Less financial means a lower price and I think when people hear the d word they assume it's like more of a regressive discrimination. There's a class system and the upper class is discriminating against the lower class to reinforce some system. I think most price discrimination is actually more egalitarian, more leveling the playing field, so to speak.
Speaker 3:Yeah, I think that's true. And again, going back to the scientific revenue days, we almost did a deal with the major social network at that point. It was a major social network that had a games network and they wanted to use us to set their prices. And they made us go out to the lead American legal system and get a letter saying that what we're doing was legal. And so we went into, okay, in the database. You cannot understand. You cannot have gender, you cannot have age, you cannot have ethnicity, you cannot. None of those things can be there. And I was like, even though reality is that Probably we'd offer discounts to senior citizens and they'd get lower prices, there's no evidence of harm here. And the lawyers were like, doesn't matter, you may not do this if you explicitly have encoded in your data set the idea of senior citizen. And I was like what if we have a proxy uses aol for email? And they were like you shouldn't have said that out loud.
Speaker 4:We use hotmail. Hotmail is now the the moniker.
Speaker 3:Yeah, but it's interesting though, because it was like, literally what we were doing, would it be illegal if we didn't coat that stuff directly? Because there are proxies in the data set that have the same outcome. It's not necessarily illegal and I was just like but there's no way we're gonna raise prices on elderly people. That's just not a function of the price sensitivity and so on, but anyway. So yeah, I think most of the time what it does is it lowers prices for people who would otherwise not buy our senior discounts technically illegal If it's a protected class, yes, it's.
Speaker 1:just nobody goes after them because no one wants to sue the movie theater for giving senior discounts.
Speaker 4:Remember they can do AARP cards, which is another way you can get around this. You can use AARP membership as another proxy because you have to be I think was it 65? 55 is AARP.
Speaker 1:Or like child discounts, like if you're under a certain age.
Speaker 3:I think that, if I understood the lawyers correctly, they are in violation, but there's a certain amount of prosecutorial discretion there right At the end of the day, am I really going to bring an enforcement action against the local movie theater so that they charge more to your grandma?
Speaker 2:There are a lot of small town diners that would go out of business for legal suits. It's interesting, like this distinction between what's right, I think from a classic kind of Reddit gamer point of view oh, it's wrong to charge people different prices and everybody needs to be on a level playing field and they have this sense that it's better. I think they think it's better for the long term health of the games and I think there's something to be said from that from an econ point of view as well. Right, you take like Costco, for example, they don't. They have this fixed margin built into their entire strategy and it's all based around longevity. Right, they could raise prices, they could improve margin, but they think that it would come at the expense of future profits or future revenue.
Speaker 2:Maybe there's something to be said there where this price, this price discrimination that's caused by profit seeking or short term profit seeking, maybe it's not good for long term revenue. I don't know if that's true and I'd be curious, bill, if you have any, what do you think? You probably have worked with more companies than all of us combined. Is that a legitimate theory? Is that?
Speaker 3:Oh, it's absolutely legitimate question right, there's two lenses. One is stability, is reassures the customer absolutely and enables certain level of predictability. And the second is the lens through which you do dynamic. Pricing is inherently a short term one. If you're looking to build an ecosystem or a game that's going to be here for the next six years and keep the players for all six years, there's always a question of like how do you know the thing you're doing today is optimizing for a year for value? Good question, right. But you can also take that a lot too far because you don't know that about anything you do today, that's a really high bar that you're only applying to one set of techniques. How do you know that game changer rolling out next month is building value on your player base and your five? You don't? Hey, you do know. Is that and this is one of the things that led to closing up shop at scientific revenue is if you do your pricing internally with employees, you know that they aren't serving five different masters at the same time and they're more thinking about the long term health of the specific game. So you free up some intellectual capacity to think about whether it's a long term good thing. It also argues for using metrics other than purely revenue metrics right. So if you're going to try a new set of prices in the AB testing terminology the idea of a guardrail right.
Speaker 3:We want to measure short term revenue change 60 day revenue change on this new set of prices Great. We are not willing to pay a price in terms of retention. Retention can't go down, okay. And we want our second purchase rate to be as high, right. Okay. So we're going to change the prices. We want to make sure that people buy a second time and a third time to the same extent they do today, because that's an indication that they're not feeling ripped off by the economic system and we want to know that they didn't just walk away from the game. And that may or may not be related to pricing, but we want to make sure that our retentions, there are engagements there, our repeat purchase rate is there and then we can. Then, if those things are all true, then we have a little bit more confidence that this pricing change probably didn't damage the health of the game in the longer term.
Speaker 2:Does that make sense? I think so. That's maybe GDP is not the only metric by which we should judge a country, so we look at per capita metrics.
Speaker 3:But the other thing that, going back to the other part of it, which was but it's an unreasonably high bar just to hold dynamic pricing techniques to, you should be applying that lens to everything. The other part of that argument that shows up sometimes is the assumption that in some sense the current prices are right and that's inherently. Even if you don't want to do dynamic pricing, even if you want to set stable prices for all eternity, how do you know you got the right ones? Interesting question Most game companies and this was more true in scientific revenue than in game data pros, but most game companies were like we set our prices by a combination of I, as a game developer, feel it's worth that, and there are some comps in the app store, and then we set it and then we don't empirically verify it, we don't change it. I was like I got zero confidence that you did that right.
Speaker 2:Zero and a price that was correct yesterday is not correct today because, exactly, exactly, it's like really quickly when you go to the.
Speaker 3:so one of my former VP of sales, ted Verani. He's now at Wapier and Wapier does a subset of what scientific revenue did, but it does global pricing. So the idea that you should charge a different price in Greece from Finland.
Speaker 4:The EU does not like that, which is again, unfortunately, like again going back to our example of welfare enhancing like this is to their own detriment, because what are you going to do? You're going to set Greece lower than Finland because their purchasing power is lower Exactly. And what happens there? Greece is going to be better off, and by enforcing one uniform price, that's worse off.
Speaker 3:That's exactly right. But Wapier has built a business out of doing the global price analysis and then recommending static pricing, but different on a geographical basis.
Speaker 4:So one of the things you mentioned is hey, I'm a product manager, I sit down and I'm trying to think about what my prices should be, and usually what I end up seeing in economy design spreadsheets is people going through IEP to hard currency conversion rates. They do the same thing in HD games as well. What is wrong with that approach?
Speaker 3:What is wrong with the approach of having a spreadsheet and?
Speaker 4:I'm doing benchmarks. That seems like a reasonable. Oh, it's a great starting point to figure out what the price should be.
Speaker 3:How do you predict tomorrow's weather? You say today's weather. Right, you can do better than that. As a starting point that industry benchmarks are a great idea. The question is, now that you've got that starting point, what do you do next? And the people who were vociferously anti-scientific revenue made the assumption that starting point was right and the setting it on industry comps was right and you should not change it, and you should not experiment, and you should. And it's what if it's wrong? And here's a great example of that from the video game industry In 2005, a console video game cost $59.99, right. In 2019, a console video game cost $59.99 because people said it by benchmarks and comps and there wasn't a lot of data to justify a change and because the community, whenever anyone tried to raise prices, would be like oh, these evil corporate bloodsuckers are out to gouge us again. And there was just such a community backlash that, like, price of a console video game didn't go up, price of a movie went up, price of a book went up.
Speaker 4:I think that's an interesting story because I think we have developed other ways to increase prices Absolutely. I think we started it.
Speaker 3:We got battle passes, we got DLC, we got the premium edition, we got like the collector's edition, where you get like the album cover art. We surreptitiously raised prices in a somewhat sneaky way so that we could maintain the baseline at $59.99. Did the price go up? Absolutely.
Speaker 4:But let me go back to the product manager for a second. Who's thinking about benchmarking their prices against other goods? So let's say I'm working at Royal Match, I'm working at Dream Games billion dollar hit new match game and I go out and I survey all these different match games. Do you think that product manager has a strong reason to believe that they should set prices or would be optimal, set prices in a different way than other match games In the case of Royal Match, but it's a particularly strong brand, right?
Speaker 3:So brand is one of, traditionally, one of those things that actually lets you raise prices. So there's that aspect. Do you think that's measurable? It's an A-B test though.
Speaker 1:You yield a mark of research about measuring brand equity.
Speaker 4:So why don't we see that though? So when I go to play these different match three games, they tend to all have the same pricing schema. Do you think that, like more indie match three games should do more price discount?
Speaker 3:So I don't play enough to know whether that statement is true. To be honest, my perception is that there are actually some variations. They're mostly on the things that aren't terribly comfortable, though A power up in Candy Crush is not the same thing as a power up in someplace else they do slightly different things.
Speaker 2:You also don't see this in comparable industries. Take the music industry. You're not paying more for a Beyonce album than some other non-important musician, right? We don't see them in movies. You don't pay more to go see a different movie of a different quality. It might be like a Marvel movie has amazing brand but you don't pay more for that. I don't know why. I don't have some theory, but within the entertainment industry as a whole there's not really like brand power, at least for that primary purchase. Now you go to the secondary market for stuff and all this like an old album, for like an old Jay-Z albums, it costs you like a bunch.
Speaker 4:It's worse than that, chris, right, because you also aren't paying differences for the seat as well. Right, I tried to implement this recently, but I pay the same amount for the front row as I do the last row, and so there are all these puzzles that I'm cannot.
Speaker 3:So the movie seats, at least in my neighborhood, they charge different prices for different seats. That one's not as true as the other parts. Music.
Speaker 4:I'm looking at I am in Sweden, by the way, so I don't know. Maybe it's an equality thing, part of pre-itunes.
Speaker 2:Different albums did cost different prices, right, and also when it was a physical but they would be like special editions right, They'd be like special editions or sound like old enough to have bought a seat.
Speaker 3:Exactly, I'm old enough to have bought vinyl and there was a scarcity thing. Right, the hot artists, there were only so many albums pressed and those were more expensive. Right, the physical goods had a wider range than iTunes. Actually, if you go back to the coverage of that, there was an extraordinary amount of Steve Jobs insisted that songs should be 99 cents, and there's a lot of coverage in the trade press at that time about music industry executives balking at this and saying, no, my artists are special, their songs are more. And ultimately they lost the argument because billions of iPods right, and the music industry has never been the same.
Speaker 3:That was probably the biggest turning point in all of music ever was the invention of the single purchase 99 cents song, and I think the point is the transition to digital goods and the transition to flat in the pricing, but it wasn't previously a part of the music industry. Movies it's fascinating. I never thought about that, but you're right. I don't know why that is.
Speaker 2:I don't know, maybe geography plays too large a role. I guess, like you pay a different price if you're in a, if you have a better movie theater, like it might like, a small movie theater in a midsize to small city in the US is going to cost much less than big, nice place in big city.
Speaker 4:And they do second tier price discrimination, because you will have time based discounts, so you will have second phase was the second round movie theaters and of course, and then it goes to HBO, which they jack up the price again, and then it hits DVD. There's a cycle here. But to your point, chris, like I don't pay more to go see Marvel's Infinity War than I do the latest indie thriller for Martin's circus.
Speaker 3:I think that boils down to a claim that the movie is only a small percentage of the experience. Yeah, which may or may not be true, if you're willing to wait 30 days, you can watch it on your TV, right, or whatever. The gap really is, and it was zero during COVID, right? And so the claim is that the process of going to the movie theater and getting the movie theater, popcorn and being in the crowd is a significant percentage of what you're paying for, and that's not different from movie to movie, increasing the price.
Speaker 4:So scientific revenue? This seems like a great thesis. You're going to go out, you're going to do some some nice words. Price dynamicism you have an SDK, you're going to try to automate a lot of it. Where does this fall apart? Why don't you think this worked out in the way you wanted it to? And I definitely would love to. I absolutely want to get into game data pros. But why don't you think this played out Like? It seems like you have one use case, like even if you can show 1% uplift on a billion dollar company, you think right, that's incredible. And all these problems are generalizable, especially regional pricing.
Speaker 3:Yeah, and I don't know that it was a bad idea. So I want to be very clear on that. Scientific revenue did not succeed. Every two to four months someone independently reinvents the idea and comes to me after having done their Googling and says why didn't it work? And I tell them, and then they go and do it anyway because that's the value of wisdom.
Speaker 4:right, you had a great talk on this, by the way that I saw I don't know Most people bad advice. Most advice is bad. Most people give advice you shouldn't care about. Is it was a good one. I want to put in the frame in that somewhere.
Speaker 3:I think that was the one to the engineering leadership. Yeah, no one listened to that talk either. Like 60 or 70 VP of engineering who wanted to be CEOs came out to be afterwards. That was a really good talk. We really needed to hear that. And whenever anyone comes up to you after talking says we really needed to hear that they didn't believe a word you said, they just thought it was useful because it helped them refine the reasons why you were wrong. So nobody took that advice either. Like Whopper is still out there, right, they're selling a variation on dynamic pricing, on global pricing, on figuring out what you should charge. Ted was my VP at scientific revenues and at Whopper, so the idea continues.
Speaker 3:But there are a bunch of things that make it hard to succeed. So, first and foremost, chris's comments about the Reddit community is people tend to get upset, regardless of whether that's justifiable or regardless of whether it's rational. If you're a game company, you depend on the goodwill of your customers in a way that airline companies just don't. If you're like oh, united Airlines is trying to screw me over, everyone's yes and that's what airline companies do. If you're like, and Amazon has publicly stated they don't do dynamic pricing, but they did. What would be the downside? It's not like people are going to stop shopping at Amazon because people make individual transaction level decisions. There's no significant moral hazard in a lot of industries. If you find out that your hotel chain is doing dynamic pricing and charging you different people, you would from price from someone else Same thing. There's not a lot of moral hazard there, whereas game companies have a slightly different position.
Speaker 3:The second thing is the long term nature, which is also a talk about conversation. How do you know you made the right decision? It's hard. The third thing is the comp of a lot of this leads people to be inherently suspicious. And then the fourth thing I would say is some combination of we had an SDK right and so now a mobile company is okay, we need to implement another SDK. It's going to sit next to the five other SDKs we have for analytics and the 17 ad SDKs we have in. Each one of them increases our crash rate by 1 to 2%.
Speaker 3:And so, like when you go to a mobile gaming company and you say we're going to do this thing, it's going to make you assuming you stick with it, assuming you go through the training cycle, assuming you let us do the things we do, you're going to make 20% more revenue. That's going to take a year because we need to train on the data set to understand the patterns. In the meantime, your engineering team is going we don't want any more SDKs, just know. And your product guys are like well, how do we know it's the right decision? It's enormously complex sales. What I'm trying to say and I think actually it is a good idea, it's just not a startup idea Because the other pressure there is. Ultimately, I raised about 10 million for a company over six years, so I got to spend $1.7 million a year. So a lot of corners. You cut on $1.7 million a year when you built Kinesity.
Speaker 4:Oh, and in California too.
Speaker 3:And so that's part of it, and the other part of it is we had major game companies do trials and they were like we realized it works. This is really interesting. Clearly you can get 13% more revenue from the game and that seems compelling, but just as clearly you're 16 people in San Mateo, california, with uncertain funding and, wow, we really don't want to rely on a lot of arguments against that right. I'm running a billion dollar franchise. Maybe I have the same concerns.
Speaker 2:So why hasn't? Before we jump into kind of the next chapter, which is GDP, why haven't we seen Google or I don't know? I guess, like game analytics would be like a smaller company but Amazon, why haven't they done something like this? Or is it a part of the other? I don't know a lot about these different kits. Does this exist?
Speaker 3:I do not know of any large scale general purpose dynamic pricing or revenue optimization systems for gaming. At this point, I don't think any exist. There are a number of small startups that like Wauperia is one of them, asatario is another. There's a company called Superwall that lets you AB test your payment walls kind of a gaming thing not, but they're all really small, right. Yeah, why hasn't there been a general purpose thing? It's a really good question and it really does lead to a little bit towards the thinking behind game data prunes and full disclosure. I pitched Apple on it in 2018, 2019. Scientific revenue not a good startup idea. My bad Colossal arrogance underestimated the size of all the problems involved and thought I could solve them all. You, apple, should be building basic pricing analytics into the ecosystem and you should be enabling people to do some of this, if not all of it, and this feels so obvious.
Speaker 4:Localized pricing. Hit a flip, flip, flip and Apple's solution proves your badging average.
Speaker 3:Tears, really. That's it. But Apple and I don't want to speak for them because it was I gave a set of talks at the spaceship on possibilities. The informal feedback I got was they did the ecosystems. Don't want to be in the position where someone can say Apple is helping the game exploit me. That third rail becomes infinitely more painful and again, that was private feedback, that wasn't the official position of Apple or anything like that, but that sort of made it a little dangerous. And then opportunity costs. They have other things to do. Right, the $3,500 3D goggles don't just invent themselves, and that I'm making fun of it a little bit. But wow, is that an enormous possible market? Yes, go do that thing.
Speaker 3:But the other part of it is if you look at the world of gaming companies I don't know how many there are really, but a very small number of them actually make significant money. Generally speaking, the ones that make significant money have been around for a while and have a lot of bespoke infrastructure and processes, and that's really one of the big problems with the scientific revenue vision. That we didn't quite understand was like if you're a large gaming company and you've got IPs and you're building these games and et cetera, et cetera. You probably have 37 database. This is the flat file system, this is the S3 system. We store that in BigQuery, we store that in Redshift oh yeah, One of the teams, one few Snowflake, and then there's like all these, and it's okay, and what we want to do is we want to take that telemetry and rationalize it into a set of covariates and features whatever your word is, your is that enable us to reason about the player population and figure out who's more likely to spend at what levels.
Speaker 3:And, by the way, we have to do so in a way that's GDPR compliant and CCPP and so on and so forth, and it becomes an enterprise software problem. At that point it's not an off the shelf product problem, it's an enterprise software problem and that's a big reason why the product hasn't emerged. You could solve all the other things, but fundamentally, the large companies in this space, their data analytics chain, they have their ways of storing the data, they have their processes in place and nobody's going to be able to walk in and say change everything you do, rewrite your infrastructure to get an 8% lift. That's just too hard and it's too risky. So ideal, you know, as you think about this. There is a body of knowledge that's starting to emerge about how to do it. There's a lot of tools out in the cloud that enable you to do pieces and parts, but I don't know that there will ever be like the equivalent of a point and click Windows MSI installer for it.
Speaker 4:What's the thinking behind Kim data pros? Take us to game data pros. Scientific revenues down. A Phoenix has emerged.
Speaker 3:Game data pros is a really interesting thing, because what happened was I shut down scientific revenue, I went through bankruptcy California law has this thing called the assignment for the benefit of creditors and what happened was I. So 2013 to 2019, six years. That was a long road, and it was ultimately a long road that wound up with a bankrupt company. I did my best to try and make it a viable, and so 2019, I'm sitting there. I'm like a former mathematician. I understand this space really well. The thing I'm going to do now is I am going to go off and become a console, because I don't want to be CEO right now. I really don't. I don't want to start another company. I don't want to know. I can't even chase the idea of being employed in a job because, wow, that was pain and, by a strange coincidence, many of the companies that weren't really interested in buying scientific revenues product we're really interested in having me consult for them on how to do things and it's okay, you won't buy my product from me. And now that company is dead and I really don't want to get a job oh, like in the nine to five cents, and I don't want to start another company. I have all. I have the scientific revenue pipeline reaching out to me and saying hey, you know, can you tell us what we're doing wrong? Can you help us understand and it. And so it's interesting. It's okay, I'll start it. I'll do some consulting, right, I'll help people understand how to do this. I'll reflect on the science lessons of scientific revenue. I'll think harder about what it is I really want to do in the long term.
Speaker 3:And then a strange thing happened, which is like a couple of them said Okay, that's really persuasive, we need to do that. We're gonna have a hard time doing it. Do you have any friends who you could like maybe hire as subcontractors and then do that thing for us? And I was like, okay, you didn't want to buy a product, but you want me to tell you how to think about the problem, and then you want me to hire my friends to do custom. So, okay, I can do this. And and now we're in 2020, right, and it's interesting, and it's a stumble, right, and and at some point they came to me and they say, like you can't really have them as subcontractors because California AB 5 and the possibility that courts might interpret them as our employees because they're working full-time on projects for us and they don't have an employment. Okay, so you want me to form a company and hire? Okay, I can do this too, and it's just like it grew out of that.
Speaker 3:But there's some really deep thinking that involved. It's involved it here as well, because I understand the market and what's going on a lot better than I did when I first did scientific revenue, and so there's also like a long-term vision of where we want to take this thing now that, more or less against my will, I was forced to create that's where GDP came from was like. There were a lot of people who were really Thought scientific revenue was compelling, but not as a product and not as a product run by this crazy guy in San Mateo with 16 employees. And so they started like pulling the knowledge from my head and they started saying, hey, you know what and I learned a lot about Everything I said about this is an enterprise sale. I learned from that.
Speaker 3:When you look at game data systems in successful game companies, it's enormously complex and the idea that you could drop an SDK in, really, when the idea that you could like automatically change not really, you can't automatically change prices. You can't drop an SDK just doesn't work at the enterprise level. And there's another thing, which is that Signets revenue is dynamic pricing, and that's a really good idea. Game data pros is revenue optimization, which is a very different idea. In a lot of ways, dynamic pricing is like a tiny little subset because there are a lot other things you should do that build your ecosystem and build a set of games as an ecosystem, and All of it needs to be informed by two things.
Speaker 3:So the first is the CTO of Accenture wrote a book, something like humans in AI or something, and he uses a metaphor of car and driver. Like you can build the car, don't build the driver that the people drive. Build them a better car. That's an important part of it. We scientific revenues goal was to automate. We step back from that a little bit again. Second part is Experimentation. Scientific revenue believed in experimentation as a means to automation and experiment price experimentation. Take GDP we believe in experimentation period far beyond a B testing, which was a B testing was what scientific revenue did.
Speaker 3:If you look at 30 years in economics and I'm just picking a time in the past I don't know when the credibility revolution really start.
Speaker 3:There's been an awful lot of really interesting work about what to do when you don't have an RCT, what to do when you don't have an AD test right, or what to do when you have an AB test but it's imperfect in some way, and a lot of that was one taking a dig abused in the economics literature to Valley, and so there's a lot of ideas that you can take and say I'm gonna build an enterprise Class experimentation framework that enables you yeah, you can run a B test and yeah, we have guard, real metrics and yeah, we have life cycle controls and stuff like that.
Speaker 3:But also we support these other ideas as well, and also, by the way, it turns out that the Classic model of failing to reject in all hypothesis is not terribly useful, if only because no one really knows what that phrase means. Some of the Bayesian models of AD testing and of inference are more suited toward what we want to do. That's emerged in the academic literature over the past 30 years in various places. It's emerged in the B school literature, and so our general approach at GDP is less about automating everything, more about experimentation, including things that go beyond AB testing, and more about the overall game, not just the Easily measured short-term revenue, but that's where, and and more about the enterprise, and that's where we wound up.
Speaker 2:The is it fair to say that GDP GDP is so scientific revenue. You had a very specific product in mind and you were trying to sell it to people. Gdp is now Companies trying to sell you on a product that they want you to build. So you're now going to them for the problems that they specifically have and trying to create custom solutions to it. And would you say that comes from the fact that you realized this isn't a generalizable industry. You can't just create some one solution fits all thing because that's that's been my experience in the industry as well.
Speaker 2:I got when we first met, though, I was working for startup Game design, a game economy design trade can be work, work for every game in existence, which was ultimately really naive and, honestly, in a way, those two they were naive and, in such a similar way, scientific revenue. I was that and my. In my experience, it really pays to just have a consultant and not try to come up with some sort of like fancy. It's much harder to sell people on a tool that you come up with that it is to let them sell you on a tool that they want you like. This is a problem.
Speaker 3:Specifically have yes and no if I were to restate that it's work. What we're talking about is Optimizing a game for some definition of optimization, for some set of metrics, and what we're really talking about is optimizing an ecosystem games because most successful game companies a multiple game and when you look at that it's very easy to say, but every one of these there's only like a hundred of game companies or 200 game companies that have sufficiently varied Ecosystem and sufficiently large amount of profits etc To make this really worth doing. And they're all different and they're different in both the large and the small. Which relational database you use? Not really a big deal. The sequel works most how the games view themselves right. Is there really a game optimization product that works for civilization 6 and Royal match? Probably not. I don't see how there could be. I'm also I am aware of my intellectual limitations. Maybe there is at the metal level.
Speaker 2:Gpt-7 will design it, possibly you send me a lot of chess puzzles. I think your cognitive level is sufficient.
Speaker 3:There's a different point here, though, which is and the chess puzzles are fun, aren't they? Oh?
Speaker 2:fun is one word. The problem is, once I start, when I can't stop until.
Speaker 3:Yeah, I realized that what I really do and I told a friend of mine this is. I wake up, I Open up Sudoku on the expert level. If I make a mistake, I drink a cup of coffee. I try again, but like at some point my brain has started for the day because I got through the expert level Sudoku Fast enough, with no mistakes, that now I can go to work.
Speaker 2:Oh yeah, that's quite the routine.
Speaker 3:I scroll through Twitter and I'm getting angry, and then I I'm ready for the day, yeah but going back though, so, like you say, there is no product Solves the problem for the wide variety of games, let alone now that we have the metaverse, now that we have, you know, new emerging Modalities of gaming itself. Absolutely true, it's not the same thing as saying it's a pure consulting plan and so there are tools that you can build. And so now you can start to say, as a game Optimization consultant, I have a tool bag of components that I carry with me. Part of it is figuring out which ones apply, absolutely consulting tasks. Part of it is integrating those tools into the underlying Systems absolutely consulting past. Part of it is advising the paint on how to use the tools properly, absolutely a consultant pass.
Speaker 3:But there's that middle piece which is the things that are in my tool bag and that can be a product story, right, and you see this replicating an enterprise software as well. The poster child for this is probably essay. Something like 70 to 80 percent Of the world's commerce flows through a system, and yet they're all different if he has a set of things that you can use, but how you use and how they're integrated. And no, we have a set of things and then you integrate them, and usually in the SAP case you hire eccentric rate them or tech in the hinders in place, like that, and then you hire essay consultants to help you optimize the thing. But it also has a large set of things that are already built, so it's it's smaller building blocks.
Speaker 2:It's not one big giant tool that does everything. It's a bunch of little tools that do specific tasks and you put this together. So we're talking about consultants, we're talking about products, we're talking about all this pretty sophisticated stuff. What's up? You're probably one of the few people in the world who has played a role in most game economists or most Economists who work in the video games industry. What are the skills that make it good at game economists, slash data scientists and, if you wouldn't mind, what is that? Is there a difference between a data scientist in the games industry and an economist? Why do we use the term game economists? There's like a bunch here to unpack, but yeah, wow.
Speaker 3:So I don't know what the difference between economist and data science is, not in a strong sense, in a weak sense of like. Previously I don't know how to draw a line there was a similar problem with like data analysts versus data scientists. Good data analysts are doing some elements of data science and good data scientists are certainly doing some element of analysis. How do you tell them apart, like the majority of what they think about? Maybe, I don't know. When you say data scientists in gaming, there are some problems that are clearly not economics. Okay, here's one churn analysis. Churn analysis is a mostly solved problem. There's a small number of fairly standard covariates and a small number of fairly standard modern Modeling approaches. But then you have to actually build the predictive system that says hey, yeah, eric's gonna churn, 70% likelihood conditional on him playing today that he's not here a week from today. That's not an economics problem. It just is, or it doesn't feel like something that the people who taught my undergraduate economics degree would find terribly important.
Speaker 2:Okay, it's like, just because you work, that it's a Revit, it's a rev ops problem, it's somebody in the revenue department, the marketing department, cares about that, but that doesn't necessarily mean that you're an economist. So would you say that all economists are data scientists but not all data scientists are economists, or is that just a false? It's a Venn diagram. There's some intersection, but the two things can be.
Speaker 3:It depends on how loose you want to be with data science, right? If you say something like economics is a science of human behavior that deals with quantitative models, then yes, deals with quantitative models. Therefore, all economists must have some sort of data-oriented background. If you said that out loud, though, then all the Marxists in all the world would say what? And you do central planning, and that's kind of data-oriented, right? So you sneak in? Not really. So there's different things there.
Speaker 3:If you went to the Austrian economics school and the road to surfdom, is there a lot of data science? Not really. On the other hand, those people are probably not designing video games. So economics overlaps with what we would call data science, and especially I alluded to the credibility revolution. And OK, we're going to use two states least squares. Ok, no, we're going to do the diff in diff and we're going to buy Scott Cunningham's book and skip past all the rap lyrics that are really annoying to the content, where he actually explains these things. Neith, I don't know if Scott probably going to be pissed at me now, but whatever.
Speaker 2:I don't know if he listens to the cast.
Speaker 3:If he does, that's great, but you get my point, though, right there's a huge amount of economics that is inherently highly quantitative.
Speaker 2:Another way to ask this question. If you go on the Game Data Pros website and you look at the job listings, the economist is not a listed like, not edgy background. It's not even something you're looking for. To get more to the question what is somebody? So I think we all know what Game Data Pros does. You're looking for a researcher it's my impression but you're also product driven. So basically, we have a lot of people who listen to this guy.
Speaker 2:We have a lot of people who are young people, who are either in a PhD or maybe they're undergrad and they're really interested in being the one to know what are the skills and typically we just say OK, keep doing your classes and learn some coding skills and learn some data, because you're going to need these types of skills that didn't necessarily teach you, especially if you're in an economics undergrad. You're not going to learn the types of skills that you would need. You're probably more likely to learn those in the computer science class or something like that than you are in an economics class. I guess, like what, you could have somebody who's super technically skilled. You could have somebody who's very inquisitive and asks really good questions. You could have somebody who's really clever and has good solutions Is there. Am I trying to? Am I trying? Am I by saying that there's some formula that makes a game economist? Or would you say that there's some?
Speaker 3:There's a lot of different things, right, and unpacking that a little bit further. You're absolutely right. We don't hire people for an economist position. Generally speaking, our customers have economists and we are providing tools To some extent economists are. Some of the time, economists are our customers as opposed to our people. That said, a fair number of our people have economics backgrounds, right? Phil hangs out with Isaac. Isaac was Ted Castronova's student, right? I think you guys had Julian Runga on here. And yes, his PhD is in marketing. But hard to tell marketing from economics. From where Isaac? Right, they're very related.
Speaker 3:We just hired Justin Smith, who's undergraduate and graduate focus towards economics Deep economics background. Our new data analyst, mason, masters in economics, right, did we hire him as an economist? No, but do we value the economic skillset Absolutely? Is it relevant? Absolutely? Now, there's a sort of distinction there, right?
Speaker 3:The second part is what skills should a good game economist and that's a hard one, because the world has changed a lot in the past 30 years Marginal cost of goods production is essentially zero. That basically renders a large part of pre-1950 discussions irrelevant. It's just true, right? If you got your PhD in the history of economic thought 1850 to 1900, it's not obvious that a lot of that other than Thorsten Berblin actually applies. Great, we have a problem and there is a lot to be said for quantitative stuff.
Speaker 3:And again, the habit of being a skeptical consumer of data who asks the hard causal inference questions is invaluable. Taking a shoe to slam at the data science industry, the number of people who are cheerleaders for the models, and, yes, you can use a deep learning base time series forecasting thing to predict that, but what did you just do? And when it says that tomorrow's outcome is dependent on these five things, is that a causal relationship? Can you actually turn those predictors into levers that you can move? Knowing that something is a predictor versus knowing something has a causal relationship to the thing being predicted is a very different question.
Speaker 4:Do you mind. So to that point, I think a lot about that. When it comes to churn models, I've seen every company first time they get their data science team up and running it's at the Tour de France. They go through certain phases, and the first phases they love to do is they love to crank out. One of these churn models Got to have a churn model. Everyone wants to see a churn model, Absolutely. It just sits there, Just sits there to have a churn model. And then there's always this mysterious question that's asked If we know someone's going to churn, then we can intervene and do something. And of course you could have done that beforehand. You can still just run an experiment and just see what the outcome is. You don't need a churn model to go ahead and do that. When they try to make that argument, we're only going to run on a subset with the people who are likely to churn Well, I never see them actually get to the point where they're actually doing anything or the output of that end up being anything significant.
Speaker 3:And there's a big question there. You can say I'm going to do some interventions to see whether I can alter churn rescue some of these people. Julian was on here. He did this in Monster World for Wuga and I think the conclusions in the published paper at least, and you should invite him back on and have him talk about it, because he knows more about his papers than I do. What they essentially discovered is that there's a lot of value in early churn prediction because you can change, but by the time your churn prediction algorithms are really confident, dude's out the door, really Nothing you can do.
Speaker 3:I think that paper's actually wrong in a subtle way that I'll explain in a moment. But that was the core finding, and so churn prediction tells you when it's a little too late. Ok, so what you really want to do is you want to find predictors of churn prediction, if you buy into that reasoning. But the other thing and this is a mistake that is made all the time throughout the entire AB testing industry. So I think that paper actually tried four different treatments, none of which was globally efficacious. But if you think of a four-armed AB test, not as trying to find the one arm, that should be your new behavior, but instead trying to find the populations for which each of those behaviors are appropriate. So I know that you're going to churn and I'm going to try. It's telling you about the different parts of the game. I'm going to try giving you a tronch of currency that'll take you two weeks to spend, because that'll reignite your enthusiasm. I'm going to try weakening, giving you more power-ups on level 784 of Candy Crush, because that's the one that kills people. Here's my five treatments, right, and you find out that none of them really work in general. But there are people for which more power-ups work and there are some people for which the tronch of currency works. And so if you view those as subgroup selection mechanisms as opposed to finding the definitive treatment, and you view it as a question in personalization and you're going to run the experiment in order to personalize the experience, in order to reduce churn, that actually has a much higher likelihood of success. It's also not finding a global remedy and it's also introducing variation in game behavior, but it has a much higher chance of success.
Speaker 3:But then, going back to Phil's original point, which is the other thing people sometimes do, we know with a fair degree of confidence that this 60% of our player base isn't going to be here 10 days from now. That feels a lot like freedom. I can do whatever the hell I want with these people and experiment wildly with game mechanics, because they were going away anyway. And so now I can try wild things and I can completely revamp the game. And the problem is one most of the time companies don't do that. And two, they never address the question of why would this generalize? Ok, so on the people who are about to read your game, you've discovered a mechanic vastly ignites their behaviors, but you don't have no idea whether that's going to work. On the people who are already and you run into this even in dynamic pricing and price testing whoa, can we lower our prices in Brazil and then use the learnings there to change our prices in the US?
Speaker 2:No, just no.
Speaker 3:There is no reason to believe that those results generalize or transfer, whatever word you want. And so, again going back to economics, someone who understands, diff and Diff is actually useful, because Diff and Diff is a tool that lets you experiment in one place and transfer to another place. Potentially, you need to satisfy the parallel trends, hypothesis, et cetera. Ultimately, the economists are the only people who really think about this problem, and that is a valuable thing as well. Not the specifics, but the why would this generalize? How do we generalize what? The approaches for doing it? Would a synthetic control make sense here? All that sort of stuff.
Speaker 1:This is more of a fun question, but obviously you've had to pitch scientific revenue a lot and I mentioned the dirty D word, discrimination, that people don't like. Have you found alternative terms to be more successful, because I know I'm sure all of us have had to pitch price discrimination to our coworkers at some point, dynamic pricing and personalization those do actually do.
Speaker 3:Personalized pricing is much friendlier than discriminatory pricing. In practice, they wind up being the same thing.
Speaker 4:I think the things that some people think about. When it comes to personalization you briefly touched on just a moment ago, bill, which is OK. I could give people more power-ups. It's so little personalization seems to take on what we traditionally think of personalization or recommended tiles or really changing the experience in a more fundamental way. I've seen it mostly just referred to as price discrimination I have better term but I'd love to see more personalization that is more consistent with how we naturally think about the term. It seems like we haven't seen enough of that.
Speaker 3:We haven't really seen much at all, because in order to personalize, you need to have a game that's parameterized. You need to be able to change the behaviors in the thing that's running on the player's device, and that often is a lot more code. Changing the price is going and editing an entry into database. Changing the fundamental mechanic is work.
Speaker 4:There's even crazy stuff that, or at least there's stuff that drives me nuts, which is I go into Call of Duty. I press multiplayer every single time. Why is that button not bigger On successive logins? It seems like the easiest personalization. Like I go through the same menus every freaking time, like why are you not just defaulting me to that or changing the size of the buttons, really not the clicks I have? We're on this. I'd love to see us go in that direction too.
Speaker 3:The other thing is I used to be an advisor to modelai and one of the things they were working on and it's still on the website, but it's not the primary focus was what you've got detailed telemetry logs. You've got players who are successful. You have a level design problem. What if, as part of your level design, you took robots that simulated successful behavior and figured out whether they would be able to do this level or explore this level, and use that as a tool inside level design, which is also not personalization in what we're talking about, but we're just saying, as we build and extend the game, we have simulations of our players that help us do this. That's also something I'd love to see a lot more of.
Speaker 4:And speaking of seeing a lot more of. Where can people find you? What if they want to learn more about Game Data Pros? How should they reach out? What's the best way to get in touch with you?
Speaker 3:It's really easy. Actually, I have a personal website. We have Game Data Pros's website. Williamgrasso at gamedataproscom will work. I go to a fair percentage of industry conferences. Interestingly enough and this is a complaint about the game economist universe Like on the game economist discord, I said, hey, I'll buy dinner for any economist who wants to participate in a meetup at GamesV and I reputed it for like GamesV summit as well Zero takers.
Speaker 4:I would say a lot of people don't go to conferences, yet it's uncommon. I never went to conferences in my professional career. You only go to. It's a shelling point for selling, and if you ain't selling anything, a lot of people don't show up for conferences, except for GDC. I think that's the one exception. Yeah, but I agree I wish there were stronger social law and I would absolutely love to have a game economist meetup at GDC, but if 100 people sign up?
Speaker 1:I'm not sure.
Speaker 3:GVP would. My company would sponsor it. I think it's a great idea, right? Let's build a community around that and let's start getting at the habit of okay, we all have different sets of knowledge, and how do we?
Speaker 4:compare notes. The conversations get way more fun for Beers Deep when we start to talk about crypto, which we embarrassingly did not do. This podcast, I think that's another hour and 30 minutes. I still am dying to know your thoughts on crypto and Web3. We need to get to auction houses either, which I'm coming about. We.
Speaker 3:I can come back. I've enjoyed the hell out of this conversation.
Speaker 2:As with all of our guests, we got to about 10% of what we had planned to get out.
Speaker 1:This was the backstory, right. We got to lay the groundwork before we get hot heated.
Speaker 2:We're going to make an announcement in the special interest group Discord chat, but we are planning on having a game economist virtual meetup. We're hoping to just get the pots stewing. But I think GDC is a great opportunity. I think we have somebody in our community who's going to be talking, so already a reason to show up. We should all do talks, because then I think your ticket gets copped. We should all just start putting forward like game economists talks. But yeah, so keep your guys' eyes out for that. Also listeners who listens to the podcast and wants to join in that conversation? But Bill's given a lot of great advice in terms of what is this special interest group, what should it be, and one of the big things was community. It's definitely something that's lacking. We want to put together a wiki. Basically, what is a game economist? What tools do they need? Because there are different tools that you need for different jobs. Not everybody needs every single tool.
Speaker 4:We can't thank you enough for spending time.
Speaker 2:Thank you for all you do.
Speaker 4:I wish you for 90 minutes and you're the CEO of a company, so we really appreciate you spending the time to talk to us my pleasure and, like I said, I'd be happy to come back.
Speaker 3:It's a fun conversation. We should teach this to our children. Economics is major, major major. Everyone has to major in economics. Number one for personal survival. Economics is major.
