Transcript (view)

0:00
bit
0:17
there are
0:26
%uh
0:34
for dud
0:43
were you guys able to catch sum up the conversation with Eric and James
0:46
in back room what you guys think too many reactions mean
0:52
into I mean money just turn to you George art
0:56
you use data a as a way to optimize what companies to
1:01
google has built its business on optimizing down a
1:06
I’m treating every single interaction by a user as a signal to do something else
1:10
so for example
1:11
I’m when lots of people click on the eighth
1:15
on listing a search query the hell with nose to lift the search query to the
1:19
first or second space
1:20
so that the user’s interactions are really harvester recycled
1:24
back into the algorithm to improve the service um
1:27
teaching a computer to translate languages is really hard
1:31
Google found a way to actually have very good feedback mechanism so
1:34
how to choose if users choose one particular translation another
1:37
tells the algorithm the value of that
1:41
the performance up that translation and so Google Translate
1:45
improves we do you guys take this idea
1:48
learning from downer in your own businesses yeah
1:52
so for us a
1:56
we hope March advertisers reach their voices on line with the machine learning
1:59
system
2:00
some up burns from as showing us what’s working and what’s not
2:05
I maybe I’m paying off a question I get asked at the end that
2:08
for for a couple years a big guy how to little players ever kind of %um
2:12
work there and there are amazing ecosystems right now
2:15
but if we just evolved in the last few years for digital advertising
2:19
where ad space
2:22
just one at a time when little rectangle at a time with a rectangle was a
2:25
your banner after a video Facebook I’m not I’m they actually put up for auction
2:29
and real-time about $35 billion times a day that we see
2:33
I’m so that’s a tremendous amount of information for us to see
2:36
and some others were by one run a luxury add electric car at your hotel
2:40
there and I just measure from yours the
2:43
where hundreds of millions of times a service at a at work here it didn’t work
2:47
there
2:47
the definitions here and there is quite divers it could be this time of day or
2:51
two and iPad
2:52
not and I’m phone or Unohana on a website that’s about cricket scores in
2:56
India
2:56
I’m whatever happens to be that’s driving the results machines can
3:00
notice that in a way that time compatible Austin skill
3:03
that time you know a group of people you know trying to study that problem
3:06
could never have done as that term is actual results
3:09
from advertisers trillion nodding this
3:13
as he speaks yeah I think we come in
3:16
at history data just one step safe from
3:19
georgians referred to it that we think what’s really important is
3:24
for organizations to be able to tap into more sources have external data that are
3:29
now available that word
3:31
was in a way available that easily 56 years ago
3:34
and if you look back on 56 years ago they were probably
3:38
less that 50 Open Data API is now they’re about
3:42
over 7,000 and if companies can harness that data and bring it into the mix with
3:48
their own private data
3:49
they have the ability to actually drive to more interesting insights
3:54
including using that data to identify some %uh the
3:58
richer opportunities as it relates to add are targeting on mobile devices
4:02
on in fact a as it relates to that particularly
4:06
a to leave us with the latest numbers show that if you can actually capture
4:11
location-based data
4:13
around individuals there’s about six hundred billion
4:16
an additional consumers spend to tap into
4:19
so we believe that one part of this problem is how do you actually
4:24
harness more sources of data to bring into the mix
4:28
with your own internal private data to be able to actually
4:32
an array and discover opportunities such can get to the floor
4:35
and that comes back to the question I think someone had in the room at the
4:39
last session at the very end
4:41
about this notion of the day to market place and being able to actually share
4:45
a lot of this external data that’s now available to determine what our
4:50
high-value trusted sources and what using and what’s not
4:53
because I think there’s no question that as the number us
4:57
sources affects while data increase into over 10,000 so there’s gonna be a lot
5:02
and that’s very during data
5:04
and not appropriate for use at the same time there’s quite a bit and that’s
5:08
highly trusted high-value data and the question is
5:11
you know how do you create a marketplace that where companies can actually share
5:15
the state out
5:16
and bring into the mix with the private data knowing what’s up high valley on
5:20
the external side
5:21
to be able to actually drive to sum up the types of targeting solutions that
5:26
George just talked about
5:27
back I’m gonna go I wanna go back to the only open data and
5:30
data market place’s first Eric you used in a very effectively
5:34
mmm telescope think so he like to exact a ted tells all your secrets
5:38
in two minutes or less right now here you’ve got we’ve got 20 minutes when
5:42
he’s not
5:43
that this is financially interest exactly come so
5:46
Bay building on the points a little earlier
5:51
one of the things that we haven’t caesar’s which is
5:54
similar to but different from what would Google has we have a massive amount up
5:58
primarily of time
5:59
so we have behavioral data about what you do in the gaming floor about what
6:02
you do in our restaurants retail shops hotels
6:05
what shows you like to see it et cetera and
6:09
we’ve been using that for years in our marketing to release make sure that the
6:12
best offer gets to their
6:14
are right customer to it said the trip for a visit what’s really changed
6:18
recently is two factors one of them
6:22
is what remote just talk about just the availability of other data
6:26
and from our standpoint we have to be very discreet very discerning about
6:29
where you actually
6:31
what did you use and where you use that because
6:34
we’ve got this implicit and in some cases explicit understanding with our
6:38
customers that
6:39
we are using the data that you volunteer to give us
6:43
com and we’re going to use that give you value and that works
6:46
when it stated that they know they’re volunteering when you start a pending
6:49
other first the data back on track to some extent
6:51
breaks down or at least get 250 right and I think that’s something that we
6:56
are being very thoughtful but as you get into the say how do you maintain
6:59
that implied contract that you gotta their relationship with the customer
7:03
while you’re
7:03
using all these other stores the data the the other thing that’s
7:06
materially change the way we do this is that the feedback loop has become
7:11
incredibly
7:12
short even for offline data so we’re used to be able to send out
7:16
were used to send out direct mail and get a response in a couple weeks there’s
7:20
any other e-mail
7:21
or a message on a mobile app in your getting responses and
7:24
in close to real time if not in real time and that gives you a lot more
7:28
ability to action the data
7:30
and you’ve had before have you found any interesting correlations
7:34
I looking at all in this of Lantana that helps you to your business
7:38
oh there’s there’s Hunza
7:42
correlations I hope you’re in the business I’m you know we we have a a
7:45
group of about 200
7:46
analyst to do nothing but sit through data and build predictive models in
7:51
particular
7:52
I an increasingly 100 people 200
7:56
data analyst for go where the best they’re mostly based in Las Vegas
8:00
we’ve got some scattered around the on
8:03
country in a couple overseas on but for the most part they’re based in Las Vegas
8:07
and they spend
8:09
they’re all time sifting through the data looking for correlations
8:12
arguing about causation versus correlations I am and
8:16
I’m building predictive models and increasingly simulations to say
8:20
what will actually happen if we take this marking action if we
8:23
change the layout at the casino for in this way so can you give us an example a
8:27
practical
8:28
real-life example if something you’ve learned that would discuss pricing from
8:31
correlation
8:33
going well I can run a great example I’m have you found out that
8:36
for example fish eaters for easy marks and suckers
8:40
but that Arians cleanup at blackjack
8:43
I’m no comment on the question how come the fifth but more broadly
8:47
masha the so
8:51
set it to that point actually we have found that
8:54
in this is a a very intuitive comment though that
8:57
won’t surprise a lot of people in the audience but if you have someone who in
9:00
our restaurants has never ever ordered remmy
9:02
you send the me offer for steak house for the picture first a
9:06
your response was going to be lower right and you can actually see that made
9:09
a difference
9:10
if they don’t know what we did two hundred people you could just ask me
9:12
that question has to be that exact
9:14
so that’s not one that is counter-intuitive one sir that is
9:16
something that you can see
9:17
you know be there is a arm lot obviously plays are
9:22
strong role and the casino industry for obvious reasons
9:25
I and it’s fairly intuitive that if you have a bad experience if you come in you
9:29
roll the dice once and
9:31
you lose all your money that you’re not gonna be having you not gonna come back
9:34
the and so we’ve known that we’ve done things historically the action that so
9:37
if you lose
9:39
in a single trip faster significantly faster than you should
9:42
you’ll often have someone approached you on the casino floor an offer to buy you
9:45
dinner offer to get tickets to the sleepy onshore
9:48
whatever it is just to make your experience our last longer so you feel
9:52
like you got more value from
9:53
seven US News the thing to do is a shout oh man it’s my first time in Vegas and
9:56
I’ve lost
9:57
a on the contrary the way you know you have to see in the data exactly so you
10:02
can’t do that this is all triggered of the data that we see in the slot
10:05
machines and table
10:06
you needed the slot machine to yell for you but what we noticed in the data was
10:10
that
10:11
small losses losses that we thought frankly we’re
10:15
not readily visible to customers on
10:18
but over a longer period of time not one trip but entrapped
10:22
if you see someone who is and we’ve got ways quantifying like a 10 percent
10:26
unlucky to not significantly unlucky
10:28
over a sustained number trapped that those people are actually significantly
10:32
more likely to defect wanna Mike
10:35
competitors them they are to say with me and so you can start to actually make
10:39
you can start a message about the thing you don’t want to do is call up and say
10:42
a body s even on market come on back great I
10:46
I’ll everything carefully but how you actually then intervene okay I’m going
10:49
to ask you about how in camouflage
10:51
yeah a sailor’s iraqi parents am
10:54
we had a CPG companies weirdest marketing pickles on mine and they were
10:57
just where they wanted a service at so
10:59
for pregnant women and I’m its fiery in the model
11:03
I’m sorry we build can have an autonomous with I’ll tell us as we build
11:07
one
11:08
the committee ones I am all for over campaigns what’s a good ones having
11:11
individuals reach campaign that’s when I have been a science to have this kind of
11:14
a
11:15
I machine but now I mean look at the machine you can sort all the patterns by
11:18
strength
11:19
and persevere how much they’re hoping how much they’re hurting it was like the
11:21
number to Wirth pattern of all time was trying to show that people sad
11:24
soprano us we figured you they don’t really care about pickles or if they do
11:28
they don’t wanna
11:29
take the time to download a coupon online out to go get them it’s more an
11:32
urgent need
11:33
I’m I can I went out to satisfy but we’ve seen other things like a
11:36
likely I
11:38
new parents about targets for part of the whole
11:42
I wasn’t more over the sea I’m sick thanks yeah I think we’re characters
11:46
and I’m I thought it was a real pattern I mentioned on our people checking for
11:50
discourse on them
11:51
I Indian newspaper sites but from the US turns out that
11:55
and ex-patriots wrecks on targets for luxury cars and I’ll sum it was kinda
11:59
popped out that and we never
12:01
politics for how do you come up with these insights to you have to sort of
12:04
ask questions on any surface on their own
12:07
and I was just looking into a red card so we
12:10
we want that the model on a budget her companion to staff assistance to show
12:14
your word cloud above
12:15
which sites at Mikes a lot since I think it’s a lot of this kind
12:19
different about and services to you know after one the biggest sites was and the
12:22
TV to comment I didn’t even know what that was at the time
12:25
going to be and enhancing times and start to notice this
12:28
this pattern I just by looking at a soda machine kinda doesn’t something
12:32
but what we found is advertisers some they’re not happy just have a black box
12:35
and for their only answer to be
12:37
that’s their boss things got better with his wife
12:40
it’s better to say things got better because you know the Spartans at work
12:44
with detective these patterns and autonomously
12:46
put it into the media plan for us and that’s why I’m we’re getting
12:49
a better sponsored so the interesting thing that I find about that his that
12:53
you as a human being that identify the parents you have the technology
12:57
kinda to deliver it in a way that a human rights a person can actually
13:01
tease it out and in the first panel there was a sphere that
13:05
people contract that the doubters can’t dictate everything and that would be no
13:09
place for human beings and that
13:10
intuition experience Jusco the wayside but here that’s actually just the
13:14
opposite human beings were essential
13:16
it with no let’s get some if she was already
13:19
constrain the campaign swayed by itself it was just that guy everything here was
13:23
to tell the story
13:25
to the customer and the customer first and take that and I mean ever going to
13:28
respond to think harder about
13:29
I’m you know the Indian or maybe probably about the Asian expatriate
13:32
audience
13:33
as targets the them because it copper you new ads for example there be more
13:37
tailored for them but in this case the machine by itself is already kinda for
13:41
are flying the plane towards that so what’s the purpose of human beings
13:45
what’s the purpose of yours that’s a deep question I’m you
13:49
seconds across executing our business class in with the students asked to the
13:52
middle
13:52
what is Buddha I thought well so similar
13:56
a its it so indifferent resumes I think I’m
13:59
you know the funny thing I i’m the hardest part above all things we’re
14:03
finding that is obvious reasons now
14:04
we can build machines that are better than people with specific disciplines
14:07
but common sense is still this thing that’s actually very hard to record
14:11
I’m so I think that this to oliver human common sense I and i think thats p.m.
14:15
you know missions are smart seeking accidentally to find a problem for them
14:18
to solving their happy to solve it
14:19
and then you realize I mean that’s actually you know hurting that to solve
14:22
that problem well-meaning I’m
14:24
in a stolen marketers for for my world the one out mister clicks
14:27
and you know machines can be great going up getting clicks but you don’t even
14:30
know if they might not be the right on its really converting
14:33
buying our products really developing a brand affinity for you
14:36
and so we’re always have the person said if I’m you know the real thing trying to
14:39
solve not something in the middle because
14:41
it’s not always clear that solving some problem kind in the Middle some proxy
14:44
really actually improve some their overall result
14:47
I think we are you know strongly believe that the human intelligence aspect of
14:52
this
14:53
house to remain even as we getting more sophisticated machine learning
14:58
techniques and systems that
15:00
bring have more bills and intelligence because what’s happening is that a lot
15:06
of the
15:06
demands of data is really moving from the back room to the front room
15:11
as you know and in the front room you’re really relying on people with the domain
15:15
expertise
15:16
to actually look at the data and know how to make the right decision
15:20
and if you try and remove them out in this equation altogether you don’t have
15:25
the assurance that the right to maine ek spread
15:27
is actually looking out the information and deducing the right answer it can be
15:32
very very dangerous
15:33
to just rely on something that a system tells you
15:37
without the demand that’s what actually having input into that and nothing’s
15:41
going to change the fact that
15:43
you know it is the humans on the front room that actually have the domain
15:46
expertise around different pockets of business
15:49
and have to play a critical role so the question i think is you know how do you
15:53
ensure that
15:54
as the system’s get smarter and bring data to the front room
15:58
you also have the right people in front of them that unnecessarily data
16:02
scientists but rather
16:03
have the right to main expertise in order to not make naive decisions
16:07
even if you take a very simple case of
16:10
some other new data a technologies out there that rely on
16:14
more sophisticated visualization techniques even that requires the right
16:18
to me an expert in front of it to make sure that you are taking away the right
16:22
now sir from that visualization
16:25
you could have someone looking at something and think that
16:29
they have found a certain answer and that data
16:32
and another person who has a different set of domain expertise come away with
16:36
the different answers so
16:37
we do believe that in order to actually ensure that as data moves the front room
16:43
and more systems come out which have to support machine learning techniques
16:47
in a great in them the domain experts just have to remain
16:50
in the equation would need to change however is their ability to actually
16:55
yet to more for community effect is
16:58
to how they look at data think you know with someone brought this up earlier
17:02
this whole notion of sharing
17:03
and not sharing data to determine what is the right data to look at and what is
17:08
not but also
17:09
when it comes the inside enabling people to collectively
17:13
come to an inside by looking at the same thing at the same time
17:17
you probably you know the reason example you know that
17:20
I’m sure some people have heard about it the Super Bowl recently
17:24
during the power outage there are of course you know
17:27
lots about the ties as always at the Super Bowl that there were four
17:30
companies in particular
17:32
that actually established marketing war rooms during the Super Bowl event itself
17:37
and during the power outage which was an unanticipated about
17:41
these four companies war literally on the edge of looking at data as it was
17:45
changing in real time to make decisions but it was their expertise
17:49
that allow them to actually take data as it was being
17:53
as it was coming live on the screen and be able to make the right decision on
17:57
what r
17:58
add to target who to target to what platforms further doubt on
18:02
already did that very well as No in town is that it’s classic example %uh
18:06
you know forty thousand dollars and yielding more than that three million
18:10
dollars
18:10
band and a matter of you know just a 15-minute period during an unanticipated
18:15
event without those
18:17
without the domain expert in the in the War Room at that time you could not have
18:22
come to that conclusion
18:23
so let me take this question and and put up a notch
18:26
and actually going to tune in just a second soviet clubhouse let’s rock a
18:30
question for Nick
18:31
so we have a war room up to 100 people at Caesars
18:37
that’s happening all the time we have some branstad for the Super Bowl creator
18:41
war room
18:42
I’m your company’s for to counterfeiters have offer services well
18:46
chinaman charges case two companies to all company see the big down a war room
18:53
such as the future just like every company has on the strategy department
18:56
I think it’s more like a you know it business going to learn how to manage
18:59
people and should learn how to manage robots
19:02
and so i i think i you these these Big Data Systems that
19:05
you try to turn on them your create parents even in some cases autonomously
19:09
take actions I’m I think it is a matter about its company
19:12
your concerning some lost so what some kinda funny reading I for one welcome
19:16
our new robot overlords but
19:18
really like it if your boss is going to be you know our workers as a matter of
19:21
learning
19:22
how to work with them so i think im here its you not everybody needs to
19:25
have the expertise internally but everyone should learn how to be a
19:28
skeptical
19:29
and informed customer these kinds of services and results
19:32
okay I think the informed consumer
19:36
day & Analytics is going to be an increasingly important role for
19:40
not just on not just people in
19:43
the analytics group but also CEO see Mo’s
19:46
CIO’s in in companies I think that the single biggest thing I see is a
19:50
talk to to other see a moes is
19:53
I that they’d the ones who are successful at this for the 12 can
19:57
consume the data well
19:58
from and actually not just understand how to apply
20:01
the domain expertise to make the action better but also understand what the date
20:05
is telling you and understand the limitations of the emails
20:08
I think there were two aspects to this is
20:11
relates to you know does everyone need to have a moral I think if you’re
20:15
looking at what we call them live situation analysis is where you have a
20:20
very short window of time to actually act
20:23
make a decision you might need these
20:26
you know small forums where people with the right expertise can see the data
20:30
very quickly and make a decision
20:32
so that you don’t have you not in danger of you know the wrong
20:36
a system making the wrong decision for you
20:39
when you’ve got a very short window so short windows being you know
20:42
a minute couple minutes five minutes ten minutes
20:45
and for that I thank you know companies will continue to have
20:49
so many top war at times when they need to act as quickly
20:53
and make the right decision looking at the data in near real-time
20:56
the other definition for a was really just the front room changing
21:01
where there’s more of a community aspect in how you look at data and breaking the
21:06
silos that have existed over the last ten years and i think that is
21:10
the other phenomena going on that we see a lot across a large
21:13
organizations now that I’ve globally distributed data teams
21:16
where what they’re saying is we need to bring these people together almost as a
21:20
virtual community where they are looking at the data
21:23
and the same day that the right time to make the decision so
21:27
is not so much a war over to community a fact that you’re creating
21:30
across the different business units and the company and breaking down the silos
21:35
verses a live situation allows as we have a very short period time to act on
21:39
and make a decision which may require certain domain experts gathered together
21:43
so love could open up the floor questions and the idea of a short period
21:47
time of course
21:48
does to the fire conference questions please
21:54
so I don’t have a first question I see a gentleman here
21:57
and then the second question %uh at
22:00
summer there’s number two please
22:04
the strongly for tree but I welcome other answers well
22:07
can you talk about how you’re using dated actually impact
22:11
the long-term behavior above the visitor as opposed to just getting him to enact
22:16
a specific transaction sir before you answer actually I’d like to
22:20
quite a couple questions cuz we arc so close to time in which one
22:23
please I see the related question again on
22:26
you spoke a little bit about the contract you have with your
22:30
to tackling Delmon earlier this morning sit back
22:34
you talked about sharing gotta with his client with you know
22:38
on people as we create a common data set
22:41
what are the limits you can wear them with that issue you know would you like
22:45
to get your clients
22:46
you back information where the limits that and how does
22:50
how does the public and the story rational decision-making
22:53
consumers rational decision-making affect how you think about
22:57
with them is a great questions I’m killing myself we have so little time
23:00
is our third question before like tariq answer all of the questions of humanity
23:06
I see those who tree can you do that in two minutes
23:09
for one minute one me see if I remember them so in terms of long-term
23:14
on effect I I think
23:18
be there is a a great tendency and and Caesars been is guilty a.m.
23:21
have this is anyone I’m I taking data
23:25
and looking at the short term impact a of the dead as you send out a piece of
23:29
mail you send out an offer and what do you get back
23:32
I’m that the real in this is where the humans actually
23:35
come in asking the right question is is important so we’ve had to consciously
23:39
take a step back and really say
23:41
I so what is the problem that we’re trying to solve if it’s a customer
23:44
profitability from
23:45
is it on the next trip or is it over two years
23:48
right because you will actually have customers whose behavior evolves over
23:52
two year period and we’d like to understand that better
23:55
so it really gets to I that there’s any
23:58
there’s a huge amount I’ve data out there and
24:01
you can you can torture that date in any sort of way the real question is
24:05
our the real artisan asking the right questions to the you get the answer
24:09
out a bit and then being willing to run the experiment so there’s a number of
24:12
examples it effective advertising
24:14
TV advertising and what-have-you on
24:18
is a really good example a the data often not showing you
24:23
the long-term impact LSU conscious the ass I questioned say
24:27
we’re going to run experiments going to be in Chicago it’s going to be and in
24:30
this zip code they’re going to see these
24:32
TV ads in this zip code they’re not and let’s look at response
24:35
over a nine-month period of time to people from those two zip
24:39
spray and you can’t do that sort of thing now with that you really couldn’t
24:43
on five years ago so you can start to ask these questions are quite
24:46
on hard longer-term you just have to have patience to
24:49
wait for the answer summit on the focus was
24:53
using information from other sources using information
24:56
so we um in 30 seconds the in the ass
25:00
caesar’s business we are very very cautious probably excessively cautious
25:04
on this point for two reasons one is that the
25:08
data that we have customers would not like %uh sharing that with other people
25:11
generally speaking
25:12
I have an example a woman who gave us her email address
25:17
five years ago for years or something like that with permission to market and
25:20
we had actually done anything with it
25:22
it turned out to give us her son’s email address not her email address and we all
25:25
this and started sending her
25:27
a local program statements to her son without knowing it
25:30
and she is understandably quite upset about the factors on now knows about her
25:35
I gambling vacation et cetera on behaviors right and that sort of thing
25:40
happens and we don’t want to cross that line so we’re probably
25:44
more cautious on this then most other businesses
25:47
out there and imagine a CpG company as it has a little less on
25:51
concern on some of these issues than we do so we have
25:54
tried to save the customer if if the customer knows about this
25:57
and they would be happy with that and we’re happy to export if the customer
26:00
find out about it and say
26:02
that violates the line then we actually stick steered clear but for the moment
26:06
yep but like to finish more about this before so we are at a time
26:09
so please join me in thanking the panelists for fastening cuffs