Amazon’s Alexa was one of the earliest assistants and gained a significant head start over its competitors. However, it has been left behind with the rise in AI. It has failed to maintain a leading position. Former senior machine learning scientist at Alexa AI shared his own perspectives on why it fell behind in the AI competition.
Mihail Eric argues that mismanagement and mislabeling of data were some of the primary reasons for the drawbacks. Alexa’s development process was hampered by stringent data protection measures. Developers found difficult to access and utilize internal data for analysis and experimentation. The available data was often poorly annotated. This led to resource-constrained environment.
Eric shared that once his team found the annotation scheme for certain data completely incorrect and this resulted in thousands of mislabeled data points. Rectifying the data involved a cumbersome process and it required approval from multiple layers of management. It was a task that was difficult by the lack of incentive for managers to pursue the changes.
Eric pointed out that fragmented organizational structure of Alexa was a significant barrier. The decentralization meant multiple small teams often worked on similar problems independently. This led to duplicated efforts and a lack of collaboration.
The misalignment between product goals and scientific research was another critical issue. The focus on customer-oriented results means that engineering and scientific efforts are to be tied to immediate product outcomes. However, it created tension with the teams which were tasked with experimental and forward-looking projects. The projects often failed to fit in a proper way into the quarterly product cycles. Hence, the teams had to justify their existence and even adjust their metrics to make it more customer-focused.
Eric recollected an incident in which his team was developing an open-domain chat system. The success metric imposed by senior leadership was scientifically unsound. It was even practically unachievable. All these led to constant conflict and eventual project abandonment.
